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BACKGROUND. 

The  Automated  Radar  Terminal  System  (ARTS)  is  a semiautomated  air  traffic 
control  system  adaptable  to  terminal  radar  facilities  of  various  densities 
and  complexities  under  a design  concept  that  permits  modular  expansion  of 
hardware  and  software  to  achieve  functional  growth  and  increased  capacity. 

Input  data  are  provided  from  airport  surveillance  radars  (ASR's)  and  air  traffic 
control  beacon  interrogators  (ATCBI’s). 

The  following  versions  of  partly  automated  radar  systems  have  been  implemented 
by  the  Federal  Aviation  Administration  (FAA)  in  terminal  area  facilities  of 
the  joint  services  under  the  generic  title  of  ARTS:  (1)  ARTS  I is  a terminal 
radar  tracking  system  installed  at  Atlanta,  Georgia.  It  tracks  primary  and 
beacon  targets  and  provides  data  blocks  and  tabular  lists.  (2)  ARTS  IA, 
presently  installed  in  the  New  York  Common  IFR  (instrument  flight  rules)  Room, 
is  a multiradar,  multiterminal  version  of  ARTS  I.  (3)  ARTS  II  is  a nontracking 
alphanumeric  modular  system  intended  for  use  at  low-density  and  medium-density 
terminals.  (4)  ARTS  III  Beacon  Tracking  Level  (BTL)  is  an  automated  system 
for  major  terminals  interfacing  with  the  National  Airspace  System  (NAS)  enroute 
radar  tracking  system  air  route  traffic  control  centers  (ARTCC's).  Alphanumeric 
tracking  data  are  provided  for  beacon  replies  only,  although  both  primary  radar 
and  beacon  data  are  displayed  in  broadband  form.  (5)  ARTS  III  Radar  Beacon 
Tracking  Level  (RBTL)  is  a further  expansion  of  ARTS  III  (BTL)  incorporated  at 
the  National  Aviation  Facilities  Experimental  Center's  (NAFEC's)  Terminal 
Automation  Test  Facility  (TATF)  at  Atlantic  City,  New  Jersey.  RBTL  tracking 
capability  was  achieved  by  the  addition  to  a basic  ARTS  III  of  a Radar  Video 
Digitizer  (RVD-3) , a second  input/output  processor  (IOP) , and  by  the  employment 
of  RBTL  operational  software  (figure  1). 

Appropriate  references  include  FAA-TD/S-120-801A,  "Federal  Aviation  Administra- 
tion Specification  for  Modularly-Expandable  ARTS  III  (TRACON  C)  Beacon  Tracking 
Levels  System;"  SPO-MD-600,  "System  Program  Office  Configuration  Management 
Directive,  ARTS  III  System  Description;"  ARTS  III  General  System  Manual;  and 
FAA-RD-74-169 , "Augmented  Radar  Beacon  Tracking  Level  System  Design  Specifica- 
tion" (reference  1,  2,  3,  and  4). 

AUTHORIZATION . 

Program  Area  Agreement  (PAA)  14-130,  "Airspace  Configuration  and  Separation 
Evaluation,"  (reference  5)  authorized  NAFEC  to  conduct  studies  of  the  various 
facets  of  the  air  traffic  control  (ATC)  system  performance  that  are  related 
to  airspace/aircraft  configuration  and  separation.  The  activities  thereunder 
dealt  with  the  ATC  terminal  radar  performance  variables  that  affect  position 
and  separation  reporting  of  aircraft,  the  reduction  and  analysis  of  data,  and 
the  display  of  results.  Previously  reported  activities  under  this  PAA  include 
FAA-RD-73-62  Parts  1,  2,  3,  "Measurement  and  Analysis  of  the  ASR-4  System 
Error"  (reference  6);  FAA-RD-74-155,  "DAIR  System  Radar  Target  Relationships" 
(reference  7);  and  FAA-RD- 73-182 , "Test  and  Evaluation  of  the  Level  1 Beacon 
Automated  Radar  Terminal  System  ARTS  III"  (reference  8) . 
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FIGURE  1.  SIMPLIFIED  BLOCK  DIAGRAM  OF  THE  RADAR  BEACON  TRACKING 
LEVEL  SYSTEM 


Activity  142-177-040,  reported  herein,  was  initiated  by  the  Terminal 
Branch,  ATC  System  Division,  System  Research  and  Development  Service 
(SRDS)  by  agreement  with  the  System  and  Equipment  Engineering  Branch,  Air 
Traffic  Systems  Division,  NAFEC.  The  TECHNICAL  APPROACH  section  of  the 
PAA  specified  that  under  this  activity,  a quantitative  assessment  of  the 
major  position  and  separation  digital  display  data  errors  would  be  con- 
ducted involving  the  use  of  live  and  simulated  aircraft  track  data  through- 
out the  geographic  airspace  of  the  terminal  area;  NAFEC  precision  tracking 
equipment  and  digital  simulation  were  to  be  employed  to  determine  the 
various  system  errors.  The  EXPECTED  PRODUCT  is  this  formal  report  of  the 
major  position  and  separation  error  components. 

OBJECTIVES. 

The  technical  objectives  of  Activity  142-177-040,  as  described  in  this 
report,  were  to  quantitatively  measure  the  positional  accuracy  of  the 
ARTS  III  digital  target  reports  and  tracking  data  and  to  analyze  the  system 
errors  for  the  purpose  of  providing  a basis  for  specifying  ATC  separation 
minima.  Specific  criteria  to  be  determined  were: 

la.  The  accuracy  to  which  ARTS  III  can  report  aircraft  positions, 

lb.  The  accuracy  to  which  ARTS  III  can  predict  (track)  aircraft  successive 

positions , 

lc.  The  accuracy  to  which  ARTS  III  can  display  aircraft  positions, 

2.  The  accuracy  to  which  ARTS  III  can  display  aircraft  separation, 

3.  The  resolution  to  which  ARTS  III  can  report  multiple  aircraft  at  various 
range  and  azimuth  positions  about  the  ASR, 

4.  The  accuracy  of  the  ARTS  III  mode  C altitude  reports,  and 

5.  The  accuracy  of  the  ARTS  III  displayed  groundspeed. 

PRODUCT  PLAN. 

A Product  Plan  (reference  9)  was  prepared  by  NAFEC  as  a technical  and  opera- 
tional document  outlining  the  effort  to  be  applied  in  the  accomplishment  of 
the  PAA.  It  related  in  detail  the  technical  approach  that  was  proposed  to 
satisfy  the  overall  objectives.  The  Product  Plan  was  provided  to  all  support 
elements  at  NAFEC  as  an  operational  document.  It  served  as  the  basis  for 
committing  resources  and  for  establishing  a time  frame  by  which  the  activity 
could  proceed.  The  end  products  to  result  from  this  activity  were  specified 
in  detail  to  ensure  that  their  form  and  content  were  both  achievable  by  support 
elements  and  satisfactory  to  the  sponsoring  agency. 


TEST  METHODOLOGY 


FACILITIES. 

NAFEC  test  facilities  allocated  to  the  activity  are  extensively  described  in 
reference  10,  "Technical  Facilities  at  NAFEC,”  excerpts  of  which  are  included 
as  an  appendix  to  this  report.  The  methodology  by  which  these  facilities  were 
employed  is  specified  in  detail  in  the  Product  Plan  and  is  illustrated  in  a 
summary  form  in  figure  2. 

The  equipments  primarily  used  to  carry  out  this  activity  were  as  follows: 

1.  The  NAFEC  ARTS  III  RBTL  TATF,  comprising 

a.  Two  input/output  processors  (IOP's), 

b.  A beacon  data  acquisition  system  (BDAS), 

c.  A radar  data  acquisition  system  (RDAS)  of  the  RVD-3  model, 

d.  Radar  Displays  with  recording  cameras, 

e.  An  RBTL  operational  program, 

f.  A 9300  card  processor  system, 

g.  A UNISERVO  Vic  magnetic  tape  subsystem,  and 

h.  A radar  beacon  simulator  (RBS)  programmable  target  generator, 

2.  An  ASR-5/ATCBI-3  radar  facility; 

3.  An  Ampex  FR-950  broadband  radar  video  recorder; 

4.  Two  NAFEC  Gulfstream  I aircraft  with  mode  3A/C  transponder  capability; 

5.  ATC  enroute  and  terminal  facilities  including  the  Atlantic  City/NAFEC 
very  high  frequency  omnirange  tactical  air  navigation  station  (VORTAC) ; 

6.  The  NAFEC  Extended  Area  Instrumentation  Radar  (EAIR)  facility; 

7.  The  NAFEC  Phototheodolite  Range  facility,  herein  referred  to  as 
Theodolites ; 

8.  An  airborne  Aircraft  Separation  Measurement  System  (ASMS); 

9.  The  NAFEC  Range  Control  Facility,  including  voice  communication  and 
data  synchronization  radio  channels; 

10.  NAFEC  data  reduction  facilities  for  photographic  and  magnetic  tape 
records;  and 


11.  A WANG  700C  Advanced  Programming  Calculator. 
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SIMPLIFIED  NAFEC  TEST  FACILITY  INTERFACE 
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CHECKOUT. 
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Several  efforts  were  involved  in  the  pretest  checkout  phase  to  ensure  the 
satisfactory  operation  of: 

1.  The  TATF  ARTS  III  RBTL, 

2.  The  RBTL  operational  software, 

3.  The  ASR  and  ATCBI , 

4.  The  ASMS  and  ATC  transponders, 

5.  The  FR-950  video  recorder, 

6.  Radar  Display  motion  picture  equipment, 

7.  TATF  UNISERVO  Vic  magnetic  tape  data  outputs, 

8.  TATF  operational  time  synchronization  to  NAFEC  Range  Control, 

9.  TATF  air-to-ground  radio  communications, 

10.  Data  processing  procedures  for  film  data, 

11.  Data  processing  procedures  for  magnetic  tape  data,  and 

12.  Data  merging  and  analysis  computer  programs. 

Considerable  time  and  effort  were  expended  to  establish  the  ARTS  III  RBTL  at 
NAFEC  and  to  bring  it  up  to  the  level  representative  of  an  operational 
facility.  A new  TATF  at  a newly  revamped  site  had  first  to  finish  construction, 
debugging,  interfacing  with  data  acquisition  sources  and  measurement  facilities, 
staffing,  training,  and  shakedown.  In  addition,  certain  hardware  and  soft- 
ware components  (RVD-3,  RBTL  operational  program)  were  received  in  an  unqual- 
ified status,  requiring  considerable  adaptation  and  modification  by  activity 
personnel.  The  magnitude  and  complexity  of  facilities  that  had  to  successfully 
interface  during  the  flight  test  programs,  coupled  with  the  fact  that  all 
facilities  were  time-shared  with  many  diverse  users,  made  maintenance  of  a 
stable  test  bed  an  extremely  demanding  task. 

CONDUCT . 

GENERAL.  Test  missions  were  scheduled  for  2 1/2-hour  durations  and  were 
conducted  on  an  average  of  three  times  a week.  The  facilities  and  manpower 
committed  to  each  mission  typically  included  two  multiengine  aircraft,  the 
EAIR  and/or  Theodolites,  the  TATF  RBTL  ARTS  III,  the  NAFEC  ASR-5  facility, 
an  FR-950  recording  site,  Radar  Display  photography,  a UNISERVO  Vic  recording 
system,  a test  control  communications  network,  and  a staff  of  engineering 
and  support  personnel. 
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Each  mission  was  preceded  by  a briefing  during  which  support  facilities  were 
advised  of  the  particulars  of  the  proposed  test;  individual  roles  were  coor- 
dinated with  the  overall  conduct  and  expected  results.  The  Product  Plan  was 
followed  as  closely  as  possible  and  served  as  the  basis  for  tying  together 
methodology  and  objectives. 

A premission  calibration  period  was  provided  during  which  support  and  test 
systems  were  configured  and  brought  online.  Radio  communications  were 
utilized  to  coordinate,  mission  activities  in  realtime,  to  ensure  readiness 
of  all  systems  prior  to  aircraft  departure,  to  overcome  contingencies,  and 
to  synchronize  data  collection  at  the  diverse  sites. 

LIVE  FLIGHT  TESTS,  CONTROL  ZONE  (Figure  3).  Missions  were  flown  with  NAFEC 
Theodolite  coverage  within  the  10-nautical-mile  (nmi)  radius  of  the  TATF. 

Two  NAFEC  aircraft  provided  target  reports  performing  maneuvers,  consisting 
of  inbound,  outbound,  converging,  diverging,  and  crossover  flights. 

LIVE  FLIGHT  TESTS,  TRANSITION  ZONE  (Figure  4).  Missions  were  flown  with  EAIR 
as  the  evaluating  facility  in  the  area  from  10  to  55  nmi  of  the  TATF.  Maneu- 
vers corresponding  to  those  described  above  were  repeated  by  the  NAFEC  air- 
craft. Beyond  25  nmi  from  NAFEC,  flight  altitudes  were  maintained  at  10,000 
feet  or  higher  to  conform  to  normal  approach  and  departure  procedures. 

LIVE  FLIGHT  TEST,  AIRCRAFT  SEPARATIONS  (Figure  5).  Missions  were  flown  with 
a NAFEC  ASMS  (Hoffman  Tactical  Air  Navigation  (TACAN)  System)  during  which 
two  NAFEC  aircraft  were  maintained  at  various  fixed  separations  from  1/2  to 
3 nmi  in  both  trail  and  abeam  formations.  In  addition,  overtake  and  convergence 
maneuvers  were  included. 

LIVE  FLIGHT  TEST,  MODE  C (Figure  6).  One  mission  in  particular  was  devoted 
to  the  collection  of  altitude  position  data  on  a NAFEC  aircraft  conducting 
descents  in  a holding  pattern  and  instrument  landing  system  (ILS)  approaches 
with  EAIR  as  the  test  standard.  Mode  C data  were  recorded  as  a general  rule 
on  all  test  flights. 

LIVE  FLIGHT  TESTS,  TRACKING  LEVELS.  Selected  live  flights  were  recorded  at 
the  ASR  on  an  Ampex  FR-950  video  recorder  to  provide  a means  of  testing  ARTS 
under  various  system  configurations  while  utilizing  a repeatable  data  source. 

It  thus  became  possible  to  compare  the  identical  aircraft  targets  with  the 
TATF  configured  as  a BTL  ARTS  III,  then  as  an  RTL  ARTS  HI,  then  lastly  as  an 
RBTL  ARTS  III.  In  addition,  observations  were  made  of  system  performance 
during  live  flight  tests  as  the  BTL  mode  of  operation  was  switched  in  and  out 
to  permit  an  assessment  of  the  contribution  of  the  RTL  mode  to  RBTL  capability. 

SIMULATION  FLIGHT  TESTS  (Figure  7).  A series  of  aircraft  target  scenarios 
(figure  8)  were  programmed  and  stored  on  a UNISERVO  Vic  tape  for  simulation 
target  generation  by  the  RBS.  Flight  plans  for  each  target  were  placed  in 
storage  in  the  ARTS  III  Data  Processing  Subsystem  (DPS)  prior  to  the  start 
of  each  test.  The  test  targets  were  then  introduced  as  programmed  on  the 
RBS  scenario  tapes.  Precise  specification  of  target  performance  and  maneuvers, 
combined  with  limitless  repeatability,  provided  high  confidence  levels  in 
results  and  efficient  use  of  facility  resources. 
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B.  HORIZONTAL  PROFILE 


FIGURE  4.  TYPICAL  AIRCRAFT  FLIGHT  PROFILE— ARTS  III  RBTL  VS.  EA1R 
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FIGURE  6 


AIRCRAFT  FLIGHT  PROFILE— MODE  C EVALUATION 
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FIGURE  7.  TATF  SYSTEM  CONFIGURATION  FOR  ARTS  III  RBTL  SIMULATION  TESTING 
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FORMAT 
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BEACON 

AND 

RADAR 

RESOLUTION 

TESTS 


000  32  targets,  4 on  each  45°  of  azimuth,  various  trail 
separations  from  1/2  to  2 1/4  rani,  radar  and  discrete 
beacon,  inbound  from  40  nmi. 

001  Same  as  000,  except  radar  only. 

002  Same  as  000,  except  separation  varies  from  2 to  3 3/4  nmi. 

003  30  targets,  flying  abeam  of  the  origin  v225,  hl80,  in 
trail  separation,  varying  from  1/2  to  2 1/2  nmi,  radar 
and  discrete  beacon. 

004  Same  as  003,  except  radar  only. 

005  Same  as  003,  except  heading  is  270°. 

006  Same  as  000,  except  minimum  azimuth  separation  is  5°, 

some  targets  radar  only. 

007  32  targets  separated  by  32  ACP's  increment  of  azimuth, 
inbound  from  40  nmi,  radar  and  discrete  beacon. 

003  32  targets  separated  by  5°  of  azimuth,  inbound  from 

40  nmi,  radar  and  discrete  beacon,  dB  level  varies  from 
0 to  31. 

^009  Same  as  008,  except  radar  (MTI)  only. 


CAPACITY 

TESTS 


TURNING 

TRACK 

TESTS 


TRACK 

SWAPS 


LIMIT 

TRACK 

SPEEDS 


”"010  192  targets,  ring  of  32  duplicated  5 times,  radar  plus 

discrete  beacon,  inbound  from  49  nmi. 

Oil  224  targets,  same  as  010,  except  duplicated  6 times. 

/ 012  256  targets,  same  as  010,  except  duplicated  7 times. 

013  192  targets,  same  as  010,  except  radar  only. 

014  224  targets,  same  as  Oil,  except  radar  only. 

015  256  targets,  same  as  012,  except  radar  only. 

r 016  Nine  targets,  188  knots,  various  turn  rates  from  0.2 
to  3.0° /s,  radar  and  discrete  beacon,  turn  centers 
about  a point  20  nmi  out  on  360°  radial. 

017  Same  as  015,  except  radar  only. 

018  Eight  targets,  180  knots,  various  turn  rates  from  3°  to 
10° /s,  radar  and  discrete  beacon,  turn  centers  about 

a point  12  nmi  out  on  360°  radial. 

S 019  Same  as  018,  except  radar  only. 

020  One  target,  accelerating  from  60  to  600  knots  (45  knots/ 
min),  climbing  from  2 to  99.9k  feet  (1,500  ft/min) , 
constant  rate  of  turn  of  2°/s,  executing  a five-turn 
spiral,  radar  and  discrete  beacon. 

021  Same  as  020,  except  radar  only. 

022  One  target,  accelerating  from  100  to  6 00  knots  (45  knot/ 
min),  climbing  from  2 to  99.9k  feet  (1,500  ft/min), 

constant  rate  of  turn  of  3°/second,  executing  a five-turn  spiral, 
radar  and  discrete  beacon. 

023  Same  as  022,  except  radar  only. 

( 024  10  targets,  converging  along  2 and/or  3 tracks  at  a 

point  20  nmi  west  of  the  origin,  in  combinations  of 
radar  plus  beacon  and  radar  only. 

f025  20  targets,  groups  of  5,  accelerating  from  0 to 

1,080  knots,  at  rates  of  45,  75,  120,  and  225  knots/min 
026  Same  as  025,  except  radar  only. 


FIGURE  8.  SIMULATION  TEST  SCENARIOS— RADAR  BEACON  SIMULATOR 
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DATA  COLLECTION.  The  results  of  each  mission  were  recorded  to  permit  offline 
analysis  (figure  2) . At  the  TATF,  a UNISERVO  Vic  magnetic  tape  of  all  target 
reports,  track  data,  and  keyboard  entries  was  produced.  Simultaneously,  a 
photographic  record  of  the  ARTS  Radar  Display  was  collected.  The  output  of 
the  ASR,  both  primary  and  beacon  replies,  was  selectively  recorded  on  an  Ampex 
FR-950.  At  each  independent  measuring  facility  (Theodolites,  EAIR,  and  the 
ASMS),  recordings  were  produced  on  magnetic  tape  to  serve  as  the  standard  for 
the  evaluation  of  the  ARTS  III  RBTL.  Observations  of  system  performance  were 
logged  online  by  monitoring  a Radar  Display  position  and  by  two-way  radio  com- 
munications with  the  test  facilities.  The  digital  tape  mode  of  data  collection 
made  possible  the  effective  use  of  computer  data  reduction  to  unpack  and  merge 

data. 

I 

FLIGHT  TEST  MISSION  REPORTS.  Subsequent  to  each  mission,  a written  report  was 
prepared  on  the  results  thereof  and  distributed  to  all  concerned  parties 
(reference  11). 

DATA  METHODOLOGY  (LIVE  FLIGHT  TESTS) 

EDITING. 

The  editing  process  consisted  of  culling  selected  portions  of  the  live  flight 
data  from  the  total  mass  of  data  generated  during  the  flights.  The  editing 
was  initially  done  automatically  by  the  data  reduction  programs.  Further 
editing  was  then  done  manually. 

AUTOMATIC  EDITING.  The  ARTS  III  generates  a reported  and  predicted  position 
report  on  every  target  encountered  during  every  radar  scan.  The  first  level 
of  editing  consisted  of  listing  only  selected  beacon  returns,  which  in  this 
case  consisted  of  the  two  project  aircraft.  These  data  were  further  edit  d 
by  selecting  only  the  best  quality  beacon  targets.  The  selection  criteria 
for  reported  target  positions  were  strong  beacon  targets  (eight  or  more  hits 
in  the  beacon  sliding  window)  with  the  highest  possible  mode  3 /t.  and  C 
validity  numbers.  Multiple  targets  within  the  same  scan  caused  by  reflec- 
tions or  "ring-around"  were  eliminated.  In  the  case  of  the  predicted  target 
positions,  only  those  containing  the  highest  track  firmness  (quality)  numbers 
were  selected.  No  automatic  editing  was  done  on  radar  quality. 

MANUAL  EDITING.  Having  culled  the  highest  quality  data  from  the  total  data 
mass,  the  next  step  was  to  analyze  these  errors  in  terms  of  the  five  project 
objectives  outlined  in  the  INTRODUCTION.  First,  the  errors  of  each  day's 

* flights  were  manually  sorted  into  data  groups  or  population  samples  repre- 

senting similar  conditions.  For  example,  data  were  grouped  according  to 
whether  the  flights  were  along  a radial  (herein  referred  to  as  straight)  or 
a tangential  ground  track.  If  straight,  a further  grouping  was  whether  the 
aircraft  was  flying  inbound  toward  the  ASR-5  or  outbound  away  from  it. 

Another  consideration  was  the  radar  quality  (RQ)  of  the  target  radar  and/or 
beacon  return.  This  radar  quality  was  indicated  by  a numerical  code  of  RQO  to 
RQ7.  RQO  indicated  no  radar  returns  at  all  from  the  target;  hence,  this  was 
a beacon-only  target.  RQ1  was  the  weakest  radar  quality,  indicating  eight  or 


less  hits  in  the  radar  sliding  window.  RQ7  was  the  strongest  radar  quality, 
indicating  19  or  more  hits  in  the  radar  sliding  window.  Beacon-only  targets 
(RQO) , for  example,  were  manually  culled  from  the  automatically-edited  data 
and  were  considered  as  one  group  of  data,  while  radar-reinforced  beacon  (merged) 
targets  showing  highest  quality  radar  returns  (RQ7)  were  likewise  extracted  and 
considered  as  another  group  of  data.  Each  day’s  flights  represented  a discrete 
data  grouping,  since  physical  and  environmental  conditions  could  conceivably 
change  from  day  to  day. 

The  procedure  in  these  analyses  was  to  manually  select  a representative  group 
for  each  discrete  population  sample,  i.e.,  one  free  of  measurement  or  record- 
ing errors.  Generally,  a group  of  from  20  to  35  was  selected.  A group  of 
this  size  was  considered  large  enough  to  adequately  define  the  required  sta- 
tistical information,  yet  was  small  enough  to  be  manageable.  The  means  and 
standard  deviations  of  the  sample  group  were  computed  by  a WANG  700C  program. 
When  25  or  less  entries  were  present  in  a sample,  an  outlier  search  (reference 
12,  page  640)  was  first  performed  on  the  data.  This  was  another  WANG  700C 
program  which  looked  for  inordinately  large  or  small  values  (outliers)  which 
statistically  could  not  be  considered  as  being  representative  of  the  data 
population.  After  discarding  any  outliers,  the  program  computed  the  mean  and 
standard  deviation  of  the  remaining  data  in  the  population  sample.  When  the 
population  sample  consisted  of  more  than  25  entries,  the  mean  and  standard 
deviation  were  computed,  and  a search  was  made  for  any  data  (errors)  which 
were  greater  than  +4  standard  deviations  from  the  mean.  Very  few  such  values 
were  found.  In  such  cases  they  were  discarded,  and  the  mean  and  standard 
deviation  were  recomputed  from  the  remaining  data.  Radar-only  targets  were 
manually  picked  from  the  ARTS  III  printout  and  processed  according  to  radar 
quality. 

REDUCTION. 


The  data  obtained  from  the  ARTS  III,  EAIR,  Theodolites,  and  ASMS  were  reduced 
by  computer  programs  and  presented  on  program  listings.  A description  of 
some  of  these  programs  and  listings  is  presented  in  appendix  B.  The  listings 
contain  a substantial  amount  of  information.  However,  in  order  to  avoid 
unnecessary  complication,  only  information  in  the  listings  which  is  germane 
to  these  investigations  was  analyzed. 

The  programs  fall  into  three  categories.  The  first  concerns  the  processing 
of  target  report  data  as  reported  by  the  ARTS  III.  The  second  concerns  the 
processing  of  target  data  as  reported  by  the  various  measurement  systems 
against  which  the  ARTS  III  reporting  accuracies  were  checked.  (These  measure- 
ment systems  consisted  of  the  Theodolites,  the  EAIR,  and  the  ASMS).  The  third 
category  concerns  the  merging  of  the  same  targets  as  reported  simultaneously 
by  the  ARTS  III  and  the  measurement  system,  and  the  determination  of  the 
range  and  azimuth  differences  or  errors. 


ARTS  III  PROGRAMS.  Two  programs  were  used  to  process  targets  reported  by  the 
ARTS  III.  These  were  the  UNIVAC  Extractor  Tape  Reduction  Program  and  the 
ARTS  III  RBTL  Unpack  Program.  A description  of  each  follows. 

UNIVAC  Extractor  Tape  Data  Reduction  Program.  This  program  is  part  of  the 
software  package  provided  with  the  ARTS  III  RBTL.  It  formats  data  concerning 
every  radar  and  beacon  target  position  report  generated  by  the  ARTS  III  as 
well  as  all  predicted  target  position  messages.  This  information  is  recorded 
on  the  UNISERVO  Vic.  A sample  of  this  printout  together  with  a detailed 
description  of  the  data  formats  is  provided  in  appendix  Bl. 

The  data  shown  on  this  listing  were  the  basic  ARTS  III  RBTL  target  position 
data  which  were  used  for  the  target  report  accuracy  analyses.  These  data 
were  used  in  other  programs  and  listings  which  are  described  below.  The  data 
on  the  radar-only  targets  were  used  directly  from  this  listing  for  the  radar- 
only  analysis. 

ARTS  III  RBTL  Unpack  Program.  This  program  provides  a listing  of  selected 
beacon  reported  and  predicted  messages.  It  operates  on  the  basic  ARTS  III 
data  in  the  UNIVAC  Extractor  Tape  Data  Reduction  Program  and  lists  the 
selected  data,  together  with  appropriate  headings.  A sample  of  this  printout 
together  with  a detailed  description  of  the  data  formats  is  provided  in 
appendix  B2. 

MEASUREMENT  SYSTEM  PROGRAMS.  Programs  were  used  to  process  target  reports 
received  by  the  EAIR  and  Theodolites,  and  also  to  process  and  record  aircraft 
separation  and  bearing  readings  as  measured  by  the  ASMS  equipment.  A 
description  of  each  follows. 

EAIR  Program.  This  program  is  incorporated  in  the  NAFEC  technical 
facilities.  It  translated  and  rotated  the  geographical  coordinates  of  the 
EAIR  so  that  the  reported  target  positions  would  be  referenced  using  the 
ASR-5  as  the  origin  rather  than  the  EAIR.  The  program  is  designed  to  record 
the  range  and  azimuth  of  a selected  beacon  target  every  0.1  second. 

Theodolite  Program.  This  program,  likewise  a part  of  the  NAFEC  technical 
facilities,  is  similar  to  the  EAIR  program.  It  recorded  the  range  and  azimuth 
of  a selected  target  as  recorded  by  a pair  of  Theodolites  every  0.1  second. 

The  geographical  coordinates  of  each  Theodolite  were  translated  and  rotated 
in  such  a manner  that  the  range  and  azimuth  of  the  target  would  be  referenced 
to  the  ASR-5  antenna  as  origin  rather  than  to  the  Theodolite  pair. 

ASMS  Program.  This  program  provides  a printout  of  the  separation  and 
bearing  between  the  two  ASMS-equipped  aircraft.  The  separation  was  in  nau- 
tical miles,  and  the  bearing  was  in  degrees  from  one  aircraft  to  the  other. 
These  values,  along  with  the  time,  were  printed  out  each  second. 

MERGE  PROGRAMS . In  order  to  determine  the  position  reporting  errors  between 
the  ARTS  III  and  the  various  equipments  against  which  they  were  compared,  it 
was  necessary  to  compare  the  range,  azimuth,  and  altitude  of  a specific 


I 


i 

i 


i 


target  as  reported  by  the  ARTS  III  with  the  corresponding  values  of  that 
specific  target  as  measured  by  measurement  standards  (i.e.,  EAIR  or  Theodo- 
lites), The  difference  between  corresponding  values  comprised  the  range, 
azimuth,  or  altitude  error  for  that  specific  target  at  a specific  instant  of 
time.  When  measuring  separation  errors  between  two  ASMS-equipped  aircraft, 
the  positional  difference  between  the  aircraft,  as  determined  by  the  ARTS  III 
program,  was  compared  against  the  separation  of  the  aircraft  determined 
directly  by  the  ASMS.  The  difference  between  these  values  comprised  the 
separation  error. 

To  do  this,  programs  were  developed  at  NAFEC  to  merge  the  ARTS  III  data  tapes 
with  those  of  the  other  equipments.  This  merging  required  that  corresponding 
target  position  reports  be  referenced  to  a common  origin  in  space  and  also  to 
a common  instant  of  time.  The  geographical  coordinates  of  the  ARTS  III/ASR-5 
antenna,  the  EAIR  antenna,  and  each  of  the  Theodolites  are  all  different. 

The  ranges,  azimuths,  and  altitudes  of  an  aircraft  in  space  will  hence  be 
referenced  to  the  coordinates  of  the  equipment  which  is  measuring  it.  Since 
the  other  equipments  were  used  for  comparison  purposes  with  the  ARTS  III,  the 
coordinates  of  these  other  equipments  were  changed  so  as  to  coincide  with  the 
ASR-5  antenna,  the  point  of  origin  of  the  ARTS  III.  This  was  done  by  trans- 
lation and  rotation  of  the  EAIR  and  Theodolite  geographic  coordinates  as 
described  above  under  MEASUREMENT  SYSTEM  PROGRAMS. 

Data  time  correlation  was  accomplished  in  a similar  manner;  whereby  the  times 
of  the  EAIR,  Theodolite,  and  ASMS  target  reports  were  synchronized  to  those 
of  the  corresponding  ARTS  III  target  reports. 

With  target  reports  from  ARTS  III  and  the  comparing  equipment(s)  based  on  a 
common  time  reference  and  geographic  point  of  origin,  position  errors  were 
computed  by  subtracting  the  range,  azimuth,  or  altitude  as  reported  by  the 
comparing  equipment  from  the  corresponding  value  as  reported  by  the  ARTS  III. 

The  merge  programs  were  processed  on  the  IBM  7090  computer.  They  were  used 
for  all  the  ARTS  III  target  reporting  accuracy  studies  except  for  the  radar- 
only  tests,  since  the  merge  programs  processed  beacon  and  rada r- reinforced 
beacon  targets  only.  Position  errors  for  the  radar-only  targets  were  obtained 
using  the  WANG  700C  Advanced  Programming  Calculator.  This  program  was  also 
used  for  the  ARTS  III  display  accuracy  investigation.  A description  of  these 
programs  follows. 


ARTS  III  Merge  Program.  This  program  was  used  to  list  the  range, 
azimuth,  and  altitude  errors  for  each  of  two  beacon-equipped  project  aircraft, 
as  well  as  their  separation  error.  The  program  could  be  used  for  comparison  of 
ARTS  III  data  with  either  EAIR,  Theodolite,  or  ASMS  data.  A common  printout  is 
used,  a sample  of  which  is  shown  in  figure  9.  It  will  be  noted  that  returns 
are  given  for  almost  every  radar  scan  (approximately  every  4.o8  seconds).  The 
project  aircraft  are  listed  as  aircraft  1 and  aircraft  2.  Foi  each  aircraft, 
the  range,  azimuth,  and  altitude  at  the  time  of  a particular  radar  scan  as 
reported  by  the  ARTS  III  are  provided  under  the  column  marked  IFF,  along  with 
the  time  of  the  scan.  It  will  be  noted  that  the  altitude  is  given  to  the  nearest 
hundred  feet,  since  this  represents  the  altitude-resolvinR  capability  of  the 
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12— pulse  mode  C transponder,  whence  the  ARTS  III  obtains  its  altitude  informa- 
tion. The  error  (ARTS  minus  EAIR,  Theodolite,  or  ASMS)  is  displayed  under  the 
ERROR  column.  This  comprises  the  raw  error  data.  Its  further  analysis  will  be 
described  in  the  DATA  ANALYSIS  section. 

WANG  700C  Program.  This  program  was  used  in  the  two  cases  where 
the  ARTS  III  merge  program  could  not  be  applied,  i.e.,  radar-only  error 
analysis  and  ARTS  III  display  error  analysis.  A sample  of  this  printout  as 
applied  to  the  radar-only  error  analysis  is  shown  in  figure  10.  Ranges  and 
azimuths  for  selected  radar-only  targets  were  first  obtained  from  the  UNIVAC 
Extractor  Tape  Data  Reduction  Program  Listing  (appendix  Bl) , along  with  the 
appropriate  sector  times.  The  corresponding  EAIR  ranges  and  azimuths  were 
obtained  from  the  EAIR  listing,  correcting  these  values  by  interpolation  to 
correspond  to  the  ARTS  III  sector  times.  Using  the  WANG  700C  Advanced 
Programming  Calculator,  the  time  of  each  scan  was  entered,  followed  by  the 
interpolated  range  and  azimuth  of  the  target  as  reported  by  the  EAIR 
(position  1)  and  the  ARTS  III  (position  2) . The  range  and  azimuth  errors 
were  automatically  computed  and  printed  out  under  the  heading  POSITION 
DIFFERENCE. 

ANALYSIS. 

Having  obtained  the  target  report  errors  by  use  of  either  the  merge  or  WANG 
programs,  the  next  step  was  to  analyze  these  errors  in  terms  of  the  five 
project  objectives. 

The  errors  from  each  discrete  data  grouping  or  sample  were  assumed  to  be 
normally  distributed.  A normal  distribution  can  be  defined  by  two  quantities 
or  parameters:  the  mean,  which  is  the  arithmetic  average  of  the  errors  in  the 
population  sample;  and  the  standard  deviation,  which  is  a numerical  indica- 
tion of  how  loosely  or  closely  the  population  sample  errors  are  clustered 
about  the  mean. 

If  the  errors  in  the  data  samples  were  shown  to  be  independent  of  any  other 
attribute  of  the  data  sample,  such  as  range  or  radar  quality,  then  the  mean 
and  standard  deviation  of  the  range  and  azimuth  errors  were  calculated  for 
each  such  data  sample.  Statistical  tests  were  then  performed  between  the 
means  and  the  standard  deviations  of  two  or  more  data  samples  to  determine 
whether  the  samples  were  representative  of  the  same  population  or  whether 
there  was  any  significant  difference  between  them. 

I 

It  was  found  that  the  samples  varied  in  physical  and  environmental  attributes; 
for  example,  the  ARTS  III/EAIR  position  accuracy  tests  were  flown  on  4 sepa- 
rate days.  Each  day's  flights  included  inbound  and  outbound  portions.  Popula- 
tion samples  within  each  day's  inbound  and  outbound  portions  were  further 
grouped  into  beacon-only  replies  and  beacon  replies  reinforced  with  high- 
quality  radar  returns  (merged).  Thus,  for  example,  the  means  and  standard 
deviations  obtained  on  each  of  the  4 days  for  the  inbound  beacon-only  target 
reports  were  compared  to  see  whether  they  could  be  considered  as  having  come 
from  the  same  population.  In  some  cases,  means  and/or  standard  deviations 
determined  to  have  come  from  the  same  population  were  pooled  or  combined;  in 
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FIGURE  10.  POSITION  ERROR  LISTING— ARTS  III  RBTL  VS.  EAIR,  RADAR-ONLY 


other  cases,  the  tests  were  used  for  comparison  purposes  between  the  popula- 
tion samples  with  differing  attributes.  For  example,  two  population  samples 
differing  in  that  one  consisted  of  beacon-only  replies  wiiile  the  other 
consisted  of  merged  replies  could  be  tested  to  see  whether  the  radar  reinforce- 
ment had  any  significant  effect  on  the  magnitude  (mean)  or  spread  (standard 
deviation)  of  the  range  or  azimuth  errors.  When  data  samples  were  pooled 
or  combined,  the  overall  sample  sizes  were  increased  from  20  or  30  up  to  as  high 
as  407. 

The  various  tests  that  were  performed  are  described  in  appendix  C of  this 
report.  The  tests  and  terminology  thereof  can  also  be  found  in  any  standard 
textbook  of  statistics.  In  comparing  data  samples,  corresponding  standard 
deviations  would  be  compared  first.  If  the  standard  deviations  compared, 
then  the  corresponding  means  were  compared.  If  the  standard  deviations  did 
not  compare,  the  corresponding  data  samples  were  not  considered  as  having 
come  from  the  same  population,  and  the  means  therefore  could  not  be  compared. 

In  that  case,  the  interval  over  which  the  actual  range  or  azimuth  errors 
occurred  was  determined. 

SIGNIFICANCE  LEVELS.  The  closer  together  corresponding  means  and  standard 
deviations  are,  the  greater  is  the  likelihood  that  they  are  from  the  same 
population.  This  likelihood  is  numerically  expressed  by  the  significance  level. 
A significance  level  of  .05  means  that  if  sampling  is  done  from  the  same 
underlying  statistical  distribution,  only  5 percent  of  all  possible  mean  or 
standard  deviation  differences  will  be  larger  than  those  calculated  using 
values  from  statistical  tables.  If  the  observed  difference  is  less  than  the 
theoretical  value,  then  it  can  be  assumed  that  the  difference  observed  was  due 
to  chance  alone.  If  the  observed  difference  is  greater  than  the  theoretical 
value,  then  it  can  be  concluded  that  something  other  than  chance  variations 
caused  the  difference  and,  in  fact,  there  are  two  distributions  or  populations 
instead  of  just  one. 

A 5-percent  significance  level  is  a tighter  or  better  fit  than  a 1-percent 
significance  level.  Generally,  a .05  level  of  significance  will  be  assumed,  and 
where  a test  of  comparison  shows  no  significant  difference  in  the  means  or 
standard  deviations  at  the  .05  level,  the  table  will  indicate  NO  SIGNIFICANT 
DIFFERENCE.  Where  a comparison  is  found  to  be  significant  at  the  .05  level, 
but  not  significant  at  the  .01  level,  the  table  will  indicate  NO  SIGNIFICANT 
DIFFERENCE  (.01).  Where  the  comparison  was  found  to  be  significant  at  the  .01 
level,  the  table  will  indicate  SIGNIFICANT  DIFFERENCE. 

TOLERANCE  LIMITS.  The  various  accuracies  to  which  the  technical  objectives 
are  addressed  may  be  interpreted  as  being  the  tolerance  limits  within  which 
a certain  proportion  of  the  individual  errors  can  be  expected  to  lie  with 
a certain  statistical  confidence.  For  this  evaluation,  these  limits  correspond 
to  a proportion  of  99.9  percent  and  a confidence  of  90  percent  (reference  9, 
page  17).  These  accuracies  are  equal  to  plus  or  minus  a certain  number,  K,  of 
standard  deviations  about  the  mean.  The  factor  K depends  on  the  sample  size 
investigated.  For  the  limits  and  probability  used  herein,  this  factor  K varies 
from  a value  of  4.3  for  a sample  size  of  20  to  a value  of  3.291  for  an  infinite 
sample  size. 
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The  individual  errors  (hence  Che  means)  are  influenced  by  biases  which  can  be 
physically  compensated  for  by  calibration,  adjustment,  etc.  These  biases  do 
not  influence  the  standard  deviations,  hence  the  accuracies  will  be  stated  as 
a function  (K)  of  standard  deviations  about  a mean  value  which,  by  physical 
adjustment,  can  be  minimized,  if  not  actually  eliminated.  Where  it  was  possible 
to  pool  or  combine  standard  deviations,  that  value  was  used  in  the  accuracy 
determinations.  Where  it  was  not  possible  to  pool  the  standard  deviations,  the 
largest  value  was  used  to  provide  the  most  conservative  accuracy  determination. 
The  results  of  the  analysis  of  the  data  in  terms  of  the  five  project  objectives 
follows. 

1.  AIRCRAFT  POSITION  REPORTING,  PREDICTING,  AND  DISPLAYING  ACCURACY.  This 
objective  actually  encompasses  three  performance  characteristics  of  the  ARTS 
III:  (1)  the  accuracy  with  which  it  can  REPORT  a digitized  radar  or  beacon 
aircraft  target,  (2)  the  accuracy  with  which  it  can  PREDICT  or  track  the 
position  of  the  aircraft,  and  (3)  the  accuracy  with  which  it  can  DISPLAY  a 
symbol  representing  that  aircraft  on  a radar  scope  or  console.  Each  of  these 
characteristics  will  be  discussed  separately. 

a.  Aircraft  Position  REPORTING  Accuracy.  This  characteristic  was  measured 
for  distances  between  3 and  10  nmi  from  the  ASR-5  by  means  of  the  two  ARTS/ 
Theodolite  flights  on  November  6 and  8,  1974.  For  distances  between  10  and 
40  nmi  from  the  ASR-5,  the  four  ARTS/EAIR  flights  of  November  18,  25,  27, 
and  December  4,  1974,  were  used.  The  straight-line  (inbound  and  outbound) 
portions  of  these  flights  were  used.  The  position  errors  (range  and  azimuth) 
were  measured  for  both  beacon-only  and  merged  (beacon  reinforced  with  good- 
quality  radar)  targets.  Two  project  aircraft  were  utilized  for  the  ARTS/ 
Theodolite  runs,  while  only  one  aircraft  was  recorded  in  the  ARTS/EAIR  flights. 

A linear  regression  analysis  (appendix  Cl)  was  performed  on  each  of  the 
range  and  azimuth  groupings  of  the  ARTS/EAIR  flights  to  determine  if  there 
was  a linear  relationship  between  the  values  of  the  errors  and  the  aircraft 
range.  This  analysis  indicated  little  or  no  correlation  of  position  errors 
with  range;  hence,  the  position  errors  in  each  data  group  were  considered 
independent  of  range.  Means  and  standard  deviations  were  accordingly  computed 
for  each  data  grouping  of  the  ARTS/Theodolite  as  well  as  the  ARTS/EAIR  flights 
(appendixes  Dl  and  D2) . 

A check  was  made  of  the  underlying  statistical  distribution  of  the  errors 
in  each  range  and  azimuth  data  grouping  of  the  above  flights.  These  distribu- 
tions were  shown  to  be  generally  normal  or  Gaussian.  Frequency  distributions 
of  the  range  and  azimuth  errors  occurring  on  December  4,  1974,  for  each  of  the 
two  target  types  are  shown  in  figures  11  and  12. 

In  addition  to  the  straight-line  flight  analyses  described  above,  posi- 
tion errors  for  the  tangential  portions  (beyond  40  nmi)  of  the  December  4, 

1974,  ARTS/EAIR  flights  were  analysed.  The  tangential  portion  connected  the 
outbound  and  inbound  radials,  occurring  somewhat  beyond  40  nmi  from  the  ASR-5. 
Finally,  position  errors  were  analysed  for  radar-only  targets  using  the  ARTS/ 
EAIR  flight  of  June  4,  1975  (figure  5).  The  June  4,  1975,  ARTS/EAIR  NAFEC- 
Hammonton  flight  was  used  for  the  radar-only  target  report  comparison  because 
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beacon  replies  were  not  received  from  one  of  the  participating  aircraft,  its 
transponder  being  intentionally  turned  OFF.  Using  the  ARTS  111  computer 
printout  of  all  ARTS  III  target  reports,  ranges  and  azimuths  of  the  radar-only 
target  reports  were  obtained  and  compared  with  the  corresponding  values  obtained 
from  the  EAIR  computer  printout.  The  radar  quality  (RQ-1  through  7)  was  also 
recorded,  and  a simple  linear  regression  analysis  was  then  performed  to  see  if 
there  was  any  correlation  between  radar  quality  and  the  magnitude  of  range  and 
azimuth  errors  for  the  radar-only  targets.  This  analysis  showed  that  there  was 
only  a fair  degree  of  negative  correlation  between  radar  quality  and  range 
error,  and  virtually  no  correlation  for  azimuth  errors.  On  the  basis  of  these 
results,  error  values  for  each  data  grouping  were  lumped  together  without 
regard  to  radar  quality.  Means  and  standard  deviations  were  computed  for  each 
grouping,  which,  in  this  case,  consisted  of  one  group  of  radar-only  targets 
with  radar  quality  (RQ=7)  and  one  group  of  merged  targets  (beacon  reinforced 
with  RQ=7  radar) . 

b.  Aircraft  Position  PREDICTING  Accuracy.  This  characteristic  was  mea- 
sured by  means  of  the  December  4,  1974,  ARTS /EAIR  flight.  The  predicted  air- 
craft positions  were  obtained  from  the  track  messages  printed  out  via  the 
ARTS  III  RBTL  Unpack  program  (appendix  B2) . These  were  obtained  for  both 
straight  (inbound  and  outbound)  and  tangential  flight  attitudes.  Radar  quality 
was  not  considered  in  this  analysis,  and  only  track  messages  of  the  highest 
degree  of  firmness  (39)  (appendix  Bl-3)  were  used.  Each  predicted  position 
error  consisted  of  the  difference  between  the  track  message  range  or  azimuth 
value  and  the  corresponding  target  position  value  as  reported  by  the  EAIR 
(i.e.,  ARTS  Ill-predicted  position  minus  EAIR-repor ted  position). 

A simple  linear  regression  analysis  was  performed  on  the  straight-flight 
groupings  to  determine  if  there  was  any  correlation  between  range  and  the 
predicted  position  errors.  Little  or  no  correlation  was  shown;  therefore, 
the  error  values  in  each  grouping  were  lumped  together  without  regard  to  range 
(appendix  D3) . These  analyses  were  performed  on  the  inbound  and  outbound 
groupings  for  both  range  and  azimuth. 

c.  Aircraft  DISPLAYING  Accuracy.  This  characteristic  was  measured  by 
means  of  flights  conducted  on  March  8,  1974.  A range  of  14  nmi  was  used. 

The  displayed  position  of  the  target  was  obtained  from  radar  display  photo- 
graphs using  the  Telereadex  machine  (figure  D-7  of  appendix  D4) . The  corres- 
ponding reported  position  was  obtained  by  an  unpack  program  similar  to  that 
described  in  appendix  B2.  The  error  was  the  displayed  minus  reported  position 
in  range  or  azimuth.  A linear  regression  analysis  was  performed  to  see  if 
there  was  any  correlation  between  the  display  errors  and  the  distance  of  the 
symbol  from  the  display  center.  This  analysis  showed  that  the  range  and 
azimuth  errors  were  correlated  with  the  distance  of  the  symbol  from  the 
display  center  (appendix  D4) . 

2.  AIRCRAFT  SEPARATION  REPORTING  ACCURACY.  This  characteristic  was  measured 
using  ARTS  III  versus  ASMS  and  ARTS  III  versus  Theodolites.  Two  project  air- 
craft were  used  in  each  method.  The  ARTS/ASMS  flights  were  performed  on 
May  30  and  June  2,  1975,  using  the  NAFEC-Hammonton  flight  pattern  (figure  5). 

The  ARTS/Theodolite  flights,  conducted  on  November  6 and  8,  1974,  were  the 
same  flights  used  for  the  target  reporting  accuracy  tests. 
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The  ARTS  III  target  report  range,  azimuth,  and  altitude  data  from  each  aircraft 
were  used  to  calculate  the  slant  range  separation  between  them.  These  separa- 
tions were  compared  to  the  corresponding  separations  as  reported  by  the  ASMS 
equipments  for  each  of  the  various  fixed  and  variable  separations.  The  dif- 
ference in  these  separations  (ARTS  minus  ASMS)  comprised  the  separation  errors. 

The  fixed  separations  consisted  of  0.5-,  1.0-,  1.5-,  and  2.0-nmi  lateral 
(abeam)  separatons  and  0.5-  and  1.5-nmi  longitudinal  (in  trail)  separations. 

The  variable  separations  consisted  of  linear  overtakes  and  angular  convergences 
with  separations  between  +3.5  nmi. 

For  the  ARTS /Theodolite  flights,  a computer  program  provided  a listing  of 
separation  in  nautical  miles  between  the  two  aircraft  at  a given  point  in 
time,  as  well  as  the  difference,  or  error,  in  the  aircraft  separation  as 
reported  by  the  Theodolites  compared  with  the  corresponding  separation  as 
reported  by  the  ARTS  III  (reference).  An  analysis  of  these  separation  errors 
was  made  for  each  of  the  four  quadrants  for  each  of  the  2 days. 

3.  AIRCRAFT  RESOLUTION  ABILITY.  This  characteristic  was  measured  by  means 
of  the  May  30,  1975,  ARTS/ASMS  flight  in  which  the  two  project  aircraft 
utilized  the  NAFEC-Hammonton  pattern  of  figure  5.  Resolution  ability  was 
measured  in  terms  of  the  percentage  of  the  total  number  of  observed  radar 
scans  where  reports  on  both  project  aircraft  were  generated  each  scan  by 
the  ARTS  III  for  any  given  range  or  azimuth  increment.  Round  reliability 
(blip/scan)  variations,  if  any,  were  ignored. 

Measurement  of  range  resolution  was  accomplished  by  utilizing  the  overtake 
flights.  The  actual  separations  of  the  two  aircraft  were  obtained  from  the 
ASMS  printout.  Resolution  percentages  were  determined  for  1/2-nmi  increments 
of  range  separations  up  to  3.5  nmi.  Only  those  scans  where  the  azimuth 
separation  of  the  aircraft  was  within  the  ATCBI-3  antenna  beamwidth  (4.5°) 
were  considered  in  the  range  resolution  measurements. 

Measurement  of  azimuth  resolution  was  accomplished  by  utilizing  all  scans  where 
the  slant  range  of  each  aircraft  was  at  least  3 nmi  from  the  ASR-5  and  where 
the  difference  in  slant  ranges  did  not  exceed  0.25  nmi.  Resolution  percentages 
were  determined  for  1°  increments  of  azimuth  separations  from  0°  to  10°. 

4.  MODE  C ALTITUDE  REPORTING  ACCURACY.  This  characteristic  was  measured  by 
means  of  the  November  6,  1974,  ARTS /Theodolite  flights.  The  ARTS  III  extracts 
and  decodes  the  mode  C altitude,  which  is  encoded  in  100-foot  increments. 

5.  DISPLAYED  GROUNDSPEED  ACCURACY.  This  characteristic  was  measured  by  means 
of  the  November  8,  1974,  ARTS /Theodolite  flight.  The  comparisons  were  made  for 
each  full  minute  common  to  both  ARTS  and  Theodolite  data  recordings  which  had 
no  reversal  of  direction  in  either  the  x or  y direction.  The  average  velocity 
for  the  ARTS  was  computed  by  averaging  the  track  velocities  printed  out  within 
the  full  minute  concerned.  These  track  velocities  were  generally  printed  out 
each  antenna  scan,  and  varied  between  10  and  13  per  minute.  The  average 
velocity  for  the  Theodolite  was  obtained  by  subtracting  the  x and  y positions  of 
the  aircraft  in  feet  at  the  end  of  the  minute  concerned  from  the  corresponding 
positions  at  the  beginning  of  the  minute.  This  was  then  converted  into  velocity. 
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RESULTS  (LIVE  FLIGHT  TESTS) 
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AIRCRAFT  POSITION  REPORTING  ACCURACY. 


RANGE  ACCURACY — STRAIGHT  FLIGHT — ARTS /FAIR.  Table  1 shows  the  means  and  stan- 
dard deviations  of  the  range  errors  for  each  of  the  two  target  types  for  each 
of  the  4 days  of  the  ARTS/EAIR  tests.  The  means  were  obtained  by  averaging 
inbound  and  outbound  means  as  described  above.  The  inbound  and  outbound  stan- 
dard deviations  for  each  target  type  for  each  day  were  statistically  compared 
by  means  of  an  F-test  (appendix  C2) . When  this  test  showed  no  significant 
differences  between  them,  the  inbound  and  outbound  standard  deviations  for 
the  day  and  target  type  concerned  were  combined  or  pooled.  This  was  the  pre- 
vailing condition  for  seven  of  the  eight  day/target-type  combinations  shown  in 
table  1.  The  one  combination  where  significant  differences  were  shown  was 
for  merged  targets  on  December  4,  1974.  For  this  case,  the  overall  standard 
deviation  was  computed  for  all  67  range  error  values  after  inbound-outbound 
biases  had  been  averaged  out. 

The  four  pooled  daily  standard  deviations  for  each  target  type  were  now  tested 
by  means  of  Bartlett's  test  for  homogeneity  of  variances  (appendix  C3) . All 
four  daily  standard  deviations  for  the  beacon-only  targets  passed  the  test  at 


TABLE  1.  RANGE  ERROR  SUMMARIES— ARTS  III  RBTL  VERSUS 
EAIR  DURING  STRAIGHT  FLIGHT 
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1.  11/25,  11/27,  and  12/04  (test  dates) 

2.  11/18,  11/25,  and  11/27  (test  dates) 
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the  .01  level  (i.e.,  there  were  no  significant  differences  between  them  at  the 
.01  level);  therefore,  they  were  pooled  into  a combined  value  of  0.028  nmi 
and  are  so  listed  in  the  COMBINED  row  of  table  1. 

i 

In  the  case  of  the  merged  targets,  the  Bartlett  test  showed  significant  dif- 
ferences between  the  four  pooled  standard  deviations.  The  Bartlett  test  was 
then  reapplied  to  the  three  highest  merged  standard  deviations  in  table  1, 
and  no  significant  difference  at  the  .01  level  was  found.  These  were  then 
combined  into  an  overall  pooled  value  of  0.023  nmi  and  are  so  shown  in  table  1. 
The  three  highest  standard  deviations  were  selected  so  as  to  give  a more 
conservative  overall  value. 

It  will  be  noted  from  table  1 that  the  means  for  November  18,  1974,  are  sub- 
stantially more  positive-going  than  the  corresponding  means  for  the  remaining 
3 days.  This  was  due  to  a bias  introduced  by  the  fact  that  a different  wire- 
strap  card  was  used  in  the  ARTS  III  on  the  November  18th  runs  than  was  used 
during  the  other  3 days  of  the  ARTS/EAIR  flights.  Since  the  means  for 
November  18  were  thus  atypical,  they  were  excluded  from  consideration  in  the 
computation  of  the  combined  means. 

An  analysis  of  variance  test  (appendix  C4)  was  performed  on  the  means  for  the 
remaining  3 days  for  the  beacon-only  targets.  Significant  differences  were 
shown  between  these  means;  hence,  they  could  not  be  pooled.  In  the  case  of 
the  merged  targets,  only  two  of  the  four  daily  means  were  available  for  con- 
sideration, since  the  mean  for  November  18  was  excluded  due  to  the  wire-strap 
card,  while  the  mean  for  December  4 was  excluded  due  to  the  fact  that  the  stan- 
dard deviation  for  that  day  was  incompatible  with  those  of  the  other  three 
days.  Therefore,  pooling  of  the  means  for  either  target  type  was  not  attempted. 
Instead,  cumulative  relative  frequency  distribution  curves  of  the  range  errors 
for  the  beacon-only  and  merged  targets  were  prepared.  These  curves  are  shown 
in  figures  13  and  14.  Inbound  and  outbound  bias  was  averaged  out  in  these 
curves.  The  errors  for  November  18  were  excluded  from  these  frequency  dis- 
tributions, since  this  was  an  atypical  day.  As  seen  from  figure  13,  99  percent 
of  the  173  range  errors  for  the  beacon-only  targets  fell  between  -0.1175  nmi 
(-714  feet)  and  +0.0425  nmi  (+258  feet),  which  is  an  interval  of  975  feet. 
Correspondingly  for  the  merged  targets,  figure  14  shows  that  99  percent  of  the 
219  range  errors  fell  between  -0.1315  nmi  (-800  feet)  and  -0.018  nmi  (-109 
feet)  which  comprises  an  interval  of  690  feet. 

RANGE  ACCURACY— STRAIGHT  FLIGHT— ARTS/THEODOLITE.  The  statistical  test  proce- 
dures shown  in  appendices  C2  and  C3  were  applied  to  the  standard  deviations  of 
the  various  data  groupings.  The  allowable  consolidations  resulted  in  an  over- 
all standard  deviation  of  0.025  nmi  for  the  beacon-only  targets  and  0.029  nmi 
for  the  merged  targets,  based  on  sample  sizes  of  407  and  238,  respectively. 

These  are  shown  in  table  2. 

The  statistical  test  procedure  shown  in  appendix  C4  (analysis  of  variance 
test)  was  used  to  determine  if  the  means  of  the  various  data  groupings  could 
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be  assumed  to  be  from  the  same  normal  distribution.  The  test  showed  signifi- 
cant differences  at  the  .01  level  of  significance;  therefore,  the  means  could 
not  be  combined.  Table  2 also  shows  the  corresponding  data  for  the  ARTS/EAIR 
flights  for  purposes  of  comparison. 

As  seen  from  table  2,  the  overall  standard  deviations  vary  from  0.023  to 
0.029  nmi  (140  to  176  feet).  The  difference  (0.006  nmi)  is  but  36  feet. 


TABLE  2.  RANGE  ERROR  INTERVALS  FOR  STRAIGHT  FLIGHT 


S tandard 

Range 

Deviation 

Target 

Interval 

(Overall ) 

Type 

(nmi) 

(nmi) 

3 to  10 

0.025 

Beacon- 

Only 

(ARTS  - THEO) 

(N=407) 

10  to  40 

0.028 

(ARTS  - EAIR) 

(N=235) 

3 to  10 

0.029 

Merged 

(ARTS  - THEO) 

(N=238) 

10  to  40 

0.023 

(ARTS  - EAIR) 

(N=215) 

Concerning  the  means,  even  with  the  bias  due  to  the  change  in  wire-strap  cards 
accounted  for,  the  remaining  ARTS/EAIR  means  and  all  ARTS /Theodolite  means 
were  determined  not  to  be  from  the  same  normal  populations;  hence,  grand 
means  could  not  be  determined. 

RANGE  ACCURACY— TANGENTIAL  VERSUS  STRAIGHT  FLIGHT.  Table  3 shows  a comparison 
of  range  errors  occurring  when  the  aircraft  was  in  a tangential  attitude 
versus  when  it  was  in  straight  flight.  This  comparison  is  given  for  both 
beacon-only  and  merged  targets.  This  study  was  made  on  the  December  4,  1974, 
flights  of  the  ARTS/EAIR  tests.  The  mean  pairs  were  tested  for  uniformity 
by  means  of  the  Aspin-Welch  test  (appendix  C5) . It  is  seen  in  table  3 that  the 
difference  in  the  means  of  the  straight  and  tangential  portions  of  the  flight 
for  beacon-only  targets  was  not  significant  at  the  .01  level,  while  in  the 
case  of  the  merged  target  means,  the  difference  was  not  significant  at  the 
.05  level.  Likewise,  the  differences  in  standard  deviations  between  straight 
and  tangential  flights  were  not  significant  at  the  .05  level  for  both  target 
types . 
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RANGE  ERROR  (ARTS  MINUS  EAIR)  IN  NAUTICAL  MILES 


FIGURE  14.  RELATIVE  FREQUENCY  DISTRIBUTION— ARTS  III  RBTL  RANGE 
ERRORS 


TABLE  3. 


RANGE  ERROR  SUMMARY  FOR  STRAIGHT  VERSUS  TANGENTIAL 
FLIGHT— ARTS  III  RBTL  VERSUS  EAIR 


Comparison:  Straight 
Versus  Tangential 

Target 

Ty£e_ 

Flight 

Altitude 

N 

(Sample 

Size) 

(Mean) 

(nmi) 

S tandard 
Deviation 
(nmi) 

Mean 

Standard 

Deviation 

Beacon- 

Only 

Straight 

63 

-0.05 

0.029 

No 

Significant 

Difference 

(0.01) 

No 

Significant 

Difference 

Tangential 

25 

-0.07 

0.024 

Merged 

Straight 

Tangential 

67 

16 

-0.08 

-0.08 

0.015 

0.019 

No 

Significant 

Difference 

No 

Significant 

Difference 

RANGE  ACCURACY— RADAR-ONLY  VERSUS  MERGED.  Table  4 shows  a comparison  of 
range  errors  between  radar-only  and  radar-reinforced  beacon  targets.  The 
radar  quality  for  each  target  type  was  the  highest  (RQ=7) . While  statistical 
significance  was  shown  between  the  means,  actually  this  difference  was  only 
0.03  nmi  (180  feet). 


TABLE  4.  RANGE  ERROR  SUMMARY  FOR  RADAR-ONLY  VERSUS 
MERGED  TARGETS 


Target 

Type 

N 

Mean 

(nmi) 

Standard 

Deviation 

(nmi) 

Comparison:  Radar-Only  Versus  Merged 

Mean 

Standard  Deviation 

Radar- 

Only 

22 

+0.02 

0.018 

Significant 

Difference 

No 

Significant 

Difference 

Merged 

25 

-0.01 

0.028 

AZIMUTH  ACCURACY- -STRAIGHT  FLIGHT — ARTS /EAIR.  Table  5 shows  the  means  and 
standard  deviations  of  the  azimuth  errors  for  each  of  the  two  target  types 
for  each  of  the  4 days  of  the  tests.  The  means  were  obtained  by  averaging 
out  the  inbound/outbound  bias  for  the  same  reason  and  in  the  same  manner  as 
was  done  for  the  range  errors.  The  daily  standard  deviations  were  likewise 
obtained  by  comparing  inbound  and  outbound  standard  deviations  for  each  target 
type  by  means  of  an  F-test  and  combining  them,  since  all  eight  pairs  showed  no 
significant  differences. 

32 


4 


TABLE  5. 


AZIMUTH  ERROR  SUMMARIES— ARTS  III  RBTL  VERSUS  EAIR 
DURING  STRAIGHT  FLIGHT 


Beacon-Only  Targets 

Merged  Targets 

Tes  t 
Date 
(1974) 

Mean 

N (degrees) 

Standard 

Deviation 

(degrees) 

N 

Mean 

(degrees) 

Standard 

Deviation 

(degrees) 

11/18 

62  1.04 

0.211 

63 

0.94 

0.176 

11/25 

61  1.00 

msm 

64 

0.91 

0.135 

11/27 

49  1.21 

B 

63 

1.03 

0.165 

12/04 

63  1.01 

0.141 

67 

0.90 

0.129 

Combined 

Significant 

Difference 

0.175 

(N=235) 

Significant 

Difference 

0. 152 
(N=257) 

The  four  pooled  daily  standard  deviations  for  azimuth  error  for  each  target 
type  were  now  tested  for  homogeneity  by  means  of  Bartlett's  test,  as  was  done 
in  the  case  of  the  range  errors.  The  values  for  the  merged  targets  passed  at 
the  .01  level;  however,  the  beacon-only  standard  deviations  failed  at  the  .01 
level.  However,  due  to  a certain  similarity  pattern  shown  by  the  standard 
deviations  of  both  target  types,  and,  inasmuch  as  the  four  pooled  daily  stan- 
dard deviations  for  the  merged  targets  passed  at  the  .01  level,  it  was  decided 
to  pool  all  4 days  of  each  target  type.  The  pooled  values  are  shown  in  the 
COMBINED  row  of  table  5. 

The  four  daily  means  for  each  target  type  were  tested  for  uniformity  by  an 
analysis  of  variance  test.  The  tests  for  both  target  types  failed,  indicating 
significant  differences  in  the  means;  hence,  the  means  could  not  be  combined. 

Cumulative  relative  frequency  distribution  curves  of  the  azimuth  errors  for 
the  beacon-only  and  merged  target  reports  were  prepared.  These  curves  are 
shown  in  figures  15  and  16.  Inbound  and  outbound  bias  was  averaged  out.  As 
seen  from  figure  15,  99  percent  of  the  azimuth  errors  for  the  beacon-only 
targets  should  fall  between  0.61°  and  1.54°,  which  is  an  interval  of  0.93°. 
Correspondingly  for  the  merged  targets,  figure  16  shows  that  99  percent  of 
the  azimuth  errors  should  fall  between  0.58°  and  1.34°,  which  comprises  an 
interval  of  0.76°. 

AZIMUTH  ACCURACY— STRAIGHT  FLIGHT — ARTS /THEODOLITES . As  in  the  case  of  the 
range  errors,  the  F-  and  Bartlett's  tests  were  applied  to  the  standard  devia- 
tions of  the  various  azimuth  data  groupings.  The  allowable  consolidations 
resulted  in  an  overall  standard  deviation  for  azimuth  errors  of  0.250°  for  the 
beacon-only  targets  and  0.171°  for  the  merged  targets  based  on  sample  sizes  of 
345  and  186,  respectively.  These  are  shown  in  table  6. 


TABLE  6.  AZIMUTH  ERROR  INTERVALS  FOR  STRAIGHT  FLIGHT 


Standard 

Range 

Deviation 

Target 

Interval 

(Overall) 

Type 

(nmi) 

(Degrees) 

3 to  10 

Beacon- 

0.250 

Only 

(ARTS  - THE0) 

(N=345) 

10  to  40 

0.175 

(ARTS/EAIR) 

(N=235) 

3 to  10 

0.171 

Merged 

(ARTS/THEO) 

(N=186) 

10  to  40 

0.152 

— 

(ARTS/EAIR) 

(N=257) 

The  statistical  test  procedure  shown  in  appendix  C4  (analysis  of  variance 
test)  was  used  to  determine  if  the  means  of  the  various  data  groupings  could 
be  assumed  to  be  from  the  same  normal  distribution.  The  test  showed  signifi- 
cant differences;  therefore,  the  means  could  not  be  combined.  Table  6 also 
shows  the  corresponding  data  for  the  ARTS/EAIR  flights  for  purposes  of  compari- 
son. 

As  seen  from  table  6,  the  overall  standard  deviations  vary  from  0.152°  to 
0.250°.  The  difference  is  less  than  0.1°.  For  both  ARTS/EAIR  and  ARTS/ 
Theodolite  data,  the  daily  azimuth  means  were  determined  not  to  be  from  the 
same  normal  populations;  hence,  grand  means  could  not  be  determined.  There- 
fore, the  tolerance  interval  about  the  mean  error  in  which  at  least  99.9 
percent  of  the  azimuth  errors  can  be  expected  to  lie  with  a 90  percent  degree 
of  confidence  is  provided  in  place  of  the  grand  means. 

AZIMUTH  ACCURACY— TANGENTIAL  VERSUS  STRAIGHT  FLIGHT.  Table  7 shows  a compari- 
son of  azimuth  errors  occurring  when  the  aircraft  was  in  a tangential  attitude 
versus  when  it  was  in  straight  flight.  This  comparison  was  made  for  both 
beacon-only  and  merged  targets.  Table  7 shows  that  for  each  of  the  two  target 
types  there  were  no  significant  differences  in  either  the  means  or  the  stan- 
dard deviations  between  straight  and  tangential  flight  attitudes. 
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AZIMUTH  ERROR  (ARTS  MINUS  EAIR)  IN  DEGREES 


76-6-] 6 


FIGURE  16.  RELATIVE  FREQUENCY  DISTRIBUTION-ARTS  III  RBTL  AZIMUTH 
ERRORS 
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TABLE  7.  AZIMUTH  ERROR  SUMMARY  FOR  STRAIGHT  VERSUS  TANGENTIAL 
FLIGHT  — ARTS/EAIR-  December  4,  1974 


Comparison:  Straight 
Versus  Tangential 

Target 

Flight 

Altitude 

N 

(Sample 

Size) 

(Mean) 

(degrees) 

Standard 
Deviation 
(degrees ) 

Mean 

Standard 

Deviation 

Beacon- 

Only 

Straight 

63 

1.01 

0.142 

No 

Significant 

Difference 

No 

Significant 

Difference 

Tangential 

25 

0.98 

0.147 

Merged 

Straight 

67 

0.90 

0.129 

No 

Significant 

Difference 

No 

Significant 

Difference 

Tangential 

16 

0.88 

0.150 

TABLE  8.  AZIMUTH  ERROR  SUMMARY  FOR  RADAR-ONLY  VERSUS 
MERGED  TARGETS 


Target 

Tyj>.e 

N 

Mean 

(degrees) 

Standard 

Deviation 

(degrees) 

Comparison:  Radar-Only  Versus  Merged 

Mean 

Standard  Deviation 

Radar- 

Only 

22 

1.15 

0.260 

Signif icant 
Difference 

No 

Significant 

Difference 

Merged 

22 

0.92 

0.234 

f 

t 


1 

A 
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AZIMUTH  ACCURACY — RADAR-ONLY  VERSUS  MERGED.  Table  8 shows  a comparison  of 
azimuth  errors  between  radar-only  and  radar-reinforced  beacon  targets.  The 


radar  quality  for  each  target  type  was  the  highest  (RQ=7).  While  statistical 
significance  was  shown  between  the  means,  actually  this  difference  was  less 
| than  0.25°. 

AIRCRAFT  POSITION  PREDICTING  ACCURACY. 

The  means  and  standard  deviations  for  the  predicted  position  errors  were  com- 
pared to  the  corresponding  values  for  the  reported  position  errors  in  table  9 
for  the  tangential  flight  situation.  For  the  straight-flight  situation,  the 
inbound-outbound  bias  for  the  predicted  position  errors  was  averaged  out, 
while  corresponding  standard  deviations  were  tested  for  uniformity  and  then 
pooled,  since  there  were  no  significant  differences  between  them.  These  pre- 
dicted position  error  values  were  compared  with  the  corresponding  reported 
position  error  values  for  both  beacon-only  and  merged  target  reports  in 
table  10. 

It  will  be  noticed  from  table  9 that  for  the  tangential  flight  situation, 
there  was  a substantial  difference  in  the  corresponding  means  and  standard 
deviations  between  the  ARTS  III  predicted  and  reported  errors  in  range.  The 
standard  deviation  (variability  about  the  mean)  of  the  predicted  range  errors 
was  0.102  nmi  (620  feet),  while  for  the  reported  range  errors,  it  was  only 
0.027  nmi.  (165  feet).  The  mean  predicted  range  error  was  -0.01  nmi  (-50 
feet),  while  for  the  reported  position,  the  mean  range  error  was  -0.07  nmi 
(-425  feet);  almost  an  order  of  magnitude  greater.  For  the  straight-flight 
situation,  table  10  also  shows  that  the  standard  deviation  of  the  predicted 
range  errors  was  still  significantly  greater  than  that  of  the  reported  range 
errors  for  both  beacon-only  and  merged  targets. 

A possible  explanation  for  the  large  difference  in  the  predicted  and  reported 
mean  range  errors  as  observed  in  table  9 (tangential  flight  situation)  is 
offered  in  figure  17.  The  aircraft  was  changing  its  course  from  an  outbound 
to  an  inbound  run.  During  part  of  this  time,  the  aircraft  was  flying  a 
counterclockwise  arc  with  respect  to  the  radar.  The  path  of  the  ARTS  III 
target  reports,  which  occur  every  scan  time,  is  shown  by  the  solid  arc  of 
figure  17.  Point  C on  this  arc  represents  an  ARTS  III  reported  position  at 
some  particular  radar  scan  time.  The  predicted  position  is  mathematically 
projected  by  the  tracking  program  from  the  history  of  several  preceding 
reported  positions.  The  predicted  paths  of  the  aircraft  is  shown  by  the  dashed 
line  in  figure  17. 

t 

Let  points  A,  B,  and  C represent  the  position  of  the  aircraft  as  reported  by 
the  EAIR , as  indicated  by  the  ARTS  III  tracking  program,  and  as  reported  by 
the  ARTS  III  circuitry,  respectively,  for  a given  point  in  time.  The  distance 
of  each  of  these  three  points  from  the  radar  site,  0,  can  be  represented  by  the 
line  segments  OA,  OB,  and  OC,  respectively. 
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TABLE  9.  REPORTED  VERSUS  PREDICTED  POSITION  ERROR  SUMMARIES— 

TANGENTIAL  FLIGHT  (ARTS/EAIR  DATA  OF  December  4,  1974) 


Error 

Category 

ARTS  III 
Reading 

Mean 

(nmi) 

Standard 

Deviation 

(nmi) 

Comparison 

Mean 

Standard 

Deviation 

Range 

Reported 

Position 

-0.07  nmi 
(N=25) 

0.027  nmi 
(N=25) 

Significant 
Dif  ference 

Significant 

Difference 

Predicted 

Position 

-0.01  nmi 
(N=42) 

0.102  nmi 
(N=42) 

Azimuth 

Reported 

Position 

0.93° 

(N=25) 

0.231° 

(N=25) 

No 

Significant 

Difference 

No 

Significant 

Difference 

Predicted 

Position 

0.83° 

(N=42) 

0.267° 

(N=42) 

TABLE  10.  REPORTED  VERSUS  PREDICTED  POSITION  ERROR  SUMMARIES— 
STRAIGHT  FLIGHT  (ARTS/EAIR  DATA  OF  December  4,  1974) 


Predicted 


Reported 

(Beacon-Only) 


Predicted 


Reported 

(Merged) 


Predicted 


Reported 

(Beacon-Only) 


Predicted 


Reported 

(Merged) 


(N=62) 


0.88° 

(N=62) 


Standard 

Deviation 


(N=62) 


0.141° 

(N=63) 


0.136° 

(N=62) 


Comparison 

Mean 

Standard 

Deviation 

Unable  to 
Compare 

Significant 

Difference 

Unable  to 
Compare 

Significant 

Difference 

Significant 

Difference 

No  Significant 
Difference 

No 

Significant 
Dif  ference 

No 

Significant 
Di f ference 
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PATH  OF  ARTS  III  TARGET  REPORTS 


— *•  PREDICTED  PATH 

• ARTS  III  REPORTED  POSITION 

□ ARTS  III  PREDICTED  POSITION 

® EAIR  TARGET  REPORTED 
POSITION 

76-6-17 


FIGURE  17.  PREDICTED  POSITION  VS.  REPORTED  POSITION  IN  TURNING  FLIGHT 
SITUATIONS 
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The  position  error  is,  by  definition,  ARTS  minus  EAIR.  Then  the  predicted 
position  error  will  be  OB  minus  OA,  or  the  line  segment  -BA.  The  corresponding 
reported  position  error  will  be  OC  minus  OA,  or  line  segment  -CA.  It  will 
be  seen  that  the  reported  error  (-CA)  is  larger  in  magnitude  (more  negative) 
than  the  corresponding  predicted  error  (-BA) . This  agrees  with  the  observed 
means  in  range  for  the  tangential  flight  situation  shown  in  table  9,  wherein 
the  reported  mean  (-0.07  nmi)  was  larger  (more  negative)  than  the  predicted 
mean  (-0.01  nmi).  In  the  case  where  the  EAIR  target  reported  position  is 
inside  the  ARTS  III  reported  position  arc,  i.e.,  ARTS  III  reports  the  aircraft 
position  further  in  range  than  it  actually  is,  the  predicted  mean  would  be 
greater  than  the  reported  mean. 

The  greater  variability  of  the  predicted  error  versus  the  reported  error  may  be 
attributed  to  the  fact  that  the  reported  target  variability  is  due  mainly  to 
system  noise;  the  predicted  error  variability  is  compounded  by  changes  in 
firmness  values  which  effect  the  position  and  velocity  of  the  predicted 
target. 

AIRCRAFT  POSITION  DISPLAYING  ACCURACY . 

Appendix  D4  shows  that  the  best  estimate  of  range  errors  between  2 and  9 inches 

from  the  display  center  varies  between  -0.07  nmi  at  2 inches  and  -0.17  nmi  at 

9 inches,  with  at  least  95  percent  of  the  individual  range  errors  lying  between 
0.01  and  -0.25  nmi.  The  best  estimate  of  azimuth  errors  between  2 and  9 inches 

from  the  display  center  varies  from  -0.07  nmi  (-1.6°)  at  2 inches  to  0.09  nmi 

(0.45°)  at  9 inches,  with  at  least  95  percent  of  the  individual  azimuth  errors 
lying  between  -0.16  and  +0.15  nmi.  The  standard  deviations  of  the  errors  from 
each  quadrant  were  pooled  to  provide  overall  estimates  of  range  and  azimuth 
error  standard  deviation  (N=40).  The  range  and  azimuth  error  standard  deviation 
estimates  were  0.03  nmi  and  0.265°,  respectively. 

AIRCRAFT  SEPARATION  REPORTING  ACCURACY . 

Standard  deviations  for  the  fixed  and  variable  separation  ASMS  flights  as  well 
as  for  the  Theodolite  flights  are  summarized  in  table  11.  Standard  deviations 
were  computed  for  the  approaching  and  receding  portion  of  each  variable  sepa- 
ration (overtake  or  angular  convergence)  flight.  The  resulting  standard 
deviations  were  tested  by  means  of  the  Bartlett's  test.  No  significant 
differences  between  them  were  found;  therefore,  they  were  pooled  into  a value 
of  0.051  nmi  and  are  so  shown  in  table  11. 

The  standard  deviations  obtained  for  the  fixed  ASMS  flights  (0.042  nmi)  and 
the  variable  separation  ASMS  flights  (0.051  nmi)  were  compared  by  the  F-test. 

No  significant  difference  was  found;  therefore,  they  were  combined  into  an 
overall  value  of  0.047  nmi  based  on  a sample  size  of  324  separation  measure- 
ments . 
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AIRCRAFT  RESOLUTION  ABILITY. 

The  resolution  percentages  are  shown  in  table  12  for  range  separation  and  in 
table  13  for  azimuth  separation  and  are  shown  graphically  in  figures  18a  and 
18b  for  range  and  azimuth  separations,  respectively.  As  seen  in  figure  18a, 
the  ARTS  III  will  successfully  resolve  95  percent  of  aircraft  separated  in 
range  (longitudinally)  by  2.5  nmi  or  more.  At  a range  separation  of  2.0  nmi 
or  more,  90  percent  of  such  aircraft  will  be  successfully  resolved.  Figure  18b 
shows  that  more  than  90  percent  of  aircraft  separated  in  azimuth  by  6°  or  more 
will  be  successfully  resolved.  Resolution  of  beacon  targets  which  overlay 
in  range  or  azimuth  is  addressed  under  Simulation  Results,  page  51. 

MODE  C ALTITUDE  REPORTING  ACCURACY. 

The  standard  deviations  of  the  altitude  errors  from  each  of  the  four  quadrants 
were  tested  by  Bartlett's  test  (appendix  C3) . No  significant  differences  were 
found;  therefore,  they  were  consolidated  into  an  overall  standard  deviation  of 
27.44  feet.  This  was  based  upon  214  measurements. 

The  means  of  the  altitude  errors  are  clustered  about  1 foot  for  all  ex.cept  one 
quadrant  (210°)  for  which  the  mean  was  28.44  feet.  An  analysis  of  variance 
test  (appendix  C4)  showed  significant  differences  between  the  210°  mean  and 
the  means  from  the  other  three  quadrants.  There  were  no  significant  differences 
between  the  other  three  means;  therefore,  they  were  combined  into  an  overall 
mean  of  1.06  feet. 

PREDICTED  GROUNDSPEED  ACCURACY . 

The  groundspeed  errors  obtained  from  the  November  8 ARTS/Theodolite  flights 
were  observed  to  have  a mean  of  4.6  knots  and  a standard  deviation  of  7.5 
knots.  This  was  based  upon  a sample  size  of  22  1-minute  intervals. 

OVERALL  POSITION  REPORTING  AND  DISPLAYING  ACCURACY . 

Based  on  the  highest  standard  deviation  estimates  of  0.029  nmi  for  position 
reporting  range  errors  and  0.030  nmi  for  radar  display  range  errors,  the  over- 
all value  of  standard  deviation  for  reporting  and  displaying  range  error 
accuracy  was  Vo.029 2 + 0.030^,  or  0.042  nmi.  In  a similar  manner,  using  the 
highest  standard  deviation  estimate  of  0.250°  for  position  reporting  azimuth 
errors  and  0.265°  for  radar  display  azimuth  errors,  the  overall  value  of 
standard  deviation  for  reporting  and  displaying  azimuth  error  accuracy  was 
0.364°. 

SUMMARY  OF  RESULTS. 

The  results  of  the  live  testing  are  summarized  below  in  terms  of  the  five 
project  objectives. 
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TABLE  12.  TARGET  RESOLUTION  PROBABILITY 
VERSUS  RANGE  SEPARATION 


Range  Separation 
In 

Nautical  Miles 


Total  Number  of 
Radar  Scans 
Observed 


Number  of  Scans  For 
Which  both  Aircraft 
Were  Reported 


Percentage 

Resolution 


These  data  were  collected  at  ranges  varying  from  approximately  3 to  20  nmi 
from  the  ASR-5  with  an  ATCBI-3  interrogator  beamwidth  of  4.5°.  Percent 
resolution  values  are  directly  related  to  the  presence  or  absence  of  both 
targets  within  the  ATCBI  beamwidth. 


TABLE  13.  TARGET  RESOLUTION  PROBABILITY 
VERSUS  AZIMUTH  SEPARATION 


Azimuth 
Separation  in 
Degrees 


Total  Number  of 
Radar  Scans 
Observed 


Number  of  Scans  For 
Which  Both  Aircraft 
Were  Reported 


Percentage 

Resolution 


AIRCRAFT  POSITION  REPORTING  ACCURACY.  For  straight-line  flights,  between  3 
and  40  nmi  from  the  ASR-5,  the  standard  deviations  of  the  range  errors  varied 
between  0.023  and  0.029  nmi  (140  and  176  feet),  while  the  standard  deviations 
of  the  azimuth  errors  varied  between  0.152°  and  0.250°.  This  included  both 
beacon-only  and  merged  (radar-reinforced  beacon)  targets.  Sample  sizes  varied 
from  186  to  407  target  reports. 

Due  to  various  physical  biases  which  manifested  themselves,  it  was  statisti- 
cally impossible  to  obtain  a grand  overall  mean  of  range  or  azimuth  errors. 
However,  based  upon  data  from  3 days  of  ARTS/EAIR  flights  (10  to  40  nmi), 

99  percent  of  the  range  errors  for  the  beacon-only  targets  occurred  between 
-0.12  and  +0.04  nmi,  while  for  the  merged  targets,  they  occurred  between 
-0.13  and  -0.02  nmi.  Using  all  4 days  of  ARTS/EAIR  flights,  99  percent  of 
the  azimuth  errors  for  the  beacon-only  targets  fell  between  0.60°  and  1.5°, 
while  for  the  merged  targets,  they  occurred  between  0.6°  and  1.35°. 

The  results  for  the  tangential  portions  of  the  flights  did  not  differ  signifi- 
cantly from  those  of  the  straight  portions. 

The  standard  deviations  for  radar-only  target  errors  did  not  differ  significant- 
ly from  those  of  the  merged  ‘'argets.  Statistically,  the  means  for  both  range 
and  azimuth  errors  of  the  radar-only  targets  significantly  differed  from  those 
of  the  merged  targets.  Actually,  however,  this  difference  was  0.03  nmi 
(180  feet)  for  the  range  error  means  and  0.23°  for  the  azimuth  error  means. 

The  radar-only  means  were  the  larger  (more  positive.) 

Based  on  the  highest  standard  deviation  estimates  of  0.029  nmi  and  0.250° 
for  range  and  azimuth  errors,  respectively,  there  is  a 90-percent  degree  of 
confidence  that  at  least  99.9  percent  of  the  target  position  reporting  errors 
will  be  within  +0.10  nmi  of  the  mean  error  in  range  and  +0.87°  of  the  mean 
error  in  azimuth. 

AIRCRAFT  POSITION  PREDICTING  ACCURACY.  It  was  found  that  the  standard  devia- 
tion of  the  range  errors  for  PREDICTED  position  errors  was  significantly  lar- 
ger than  that  for  the  REPORTED  position  errors.  This  was  especially  true  for 
the  tangential  flight  situation,  where  the  standard  deviation  of  the  range 
error  for  the  PREDICTED  position  was  0.102  nmi,  while  for  the  REPORTED  position, 
it  was  only  0.027  nmi.  The  magnitude  of  the  mean  reported  range  error  was 
much  greater  than  that  of  the  mean  predicted  range  error  (-0.07  nmi  versus 
-0.01  nmi) . 

Based  on  the  standard  deviation  estimates  of  0.102  nmi  in  range  and  0.267° 
in  azimuth,  respectively,  for  tangential  flight,  there  is  a 90-percer.t  degree 
of  confidence  that  at  least  99.9  percent  of  the  target  position  prediction 
errors  will  fall  within  +0.40  nmi  of  the  mean  error  in  range  and  +1.04°  of  the 
mean  error  in  azimuth. 

Based  on  the  corresponding  standard  deviation  estimates  of  0.045  nmi  and 
0.136°,  respectively,  for  straight  flight,  the  corresponding  tolerance  limits 
are  +0.17  nmi  and  +0.51°,  respectively. 
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AIRCRAFT  POSITION  DISPLAYING  ACCURACY.  Range  and  azimuth  errors  were  corn 
lated  with  the  distance  of  the  symbol  from  the  display  center,  between  2 and 
9 inches  from  the  display  center,  range  error  varied  between  -0.07  and  -0.17 
nmi,  and  azimuth  error  varied  from  -0.07  to  +0.09  nmi  for  a 14-nmi  range 
setting.  At  least  95  percent  of  the  individual  range  errors  should  lie 
between  +0.01  and  -0.25  nmi,  while  at  least  95  percent  of  the  individual 
azimuth  errors  should  lie  between  -0.16  and  +0.15  nmi.  The  display  range 
errors  showed  a mean  of  -0.12  nmi,  with  a standard  deviation  of  0.03  nmi.  rhe 
display  azimuth  errors  showed  a mean  of  0.13°,  with  a standard  deviation  of 
0.265°. 

Based  on  standard  deviation  estimates  of  0.03  nmi  in  range  and  0.265°  in 
azimuth,  there  is  a 90-percent  degree  of  confidence  that  at  least  99.9  percent 
of  the  radar  display  errors  will  fall  within  +0.12  nmi  of  the  mean  range  error 
and  +1.04°  of  the  mean  azimuth  error.  A more  complete  discussion  of  display 
error  is  presented  in  appendix  D4. 

AIRCRAFT  SEPARATION  REPORTING  ACCURACY.  The  ARTS/ASMS  separation  errors  for 
both  fixed  and  variable  aircraft  separations  were  found  to  have  a combined 
standard  deviation  of  0.047  nmi.  Based  on  this  estimate  of  standard  deviation, 
there  is  a 90-percent  degree  of  confidence  that  at  least  99.9  percent  of  the 
target  report  separation  errors  will  fall  within  +0.163  nmi  of  the  mean  error. 

AIRCRAFT  RESOLUTION  ABILITY.  As  shown  by  the  resolution  investigations,  the 
ARTS  III  should  successfully  resolve  90  percent  of  aircraft  separated  in 
range  (longitudinally)  by  2.00  nmi  or  more  and  separated  in  azimuth  (laterally) 
by  6°  or  more. 

HOPE  C ALTITUDE  REPORTING  ACCURACY.  The  overall  standard  deviation  for  alti- 
tude errors  (ARTS  III  mode  C versus  Theodolite)  was  27.44  feet.  The 
corresponding  overall  mean  was  1.06  feet. 

Based  on  the  standard  deviation  estimate  of  27.44  feet,  there  is  a 90-percent 
degree  of  confidence  that  at  least  99.9  percent  of  the  mode  C altitude  report 
errors  will  be  within  +96  feet  of  the  mean  error. 

DISPLAYED  GROUNPSREED  ACCURACY.  Average  ARTS  versus  Theodolite  groundspeed 
error  was  4.6  knots,  with  a standard  deviation  of  7.5  knots. 

Based  on  the  standard  deviation  estimate  of  7.5  knots,  there  is  a 90-percent 
degree  of  confidence  that  at  least  99.9  percent  of  the  groundspeed  prediction 
errors  will  be  within  +32  knots  of  the  mean  error. 

OVERALL  ARTS  III  POSITION  REPORTING  AND  DISPLAYING  ACCURACY.  Based  on  the 
standard  deviation  estimates  of  0.042  nmi  for  range  errors  and  0.364°  for 
azimuth  errors,  there  is  a 90-percent  degree  of  confidence  that  at  least 
99.9  percent  of  the  target  position  reporting  and  displaying  errors  will  fall 
within  +0.16  nmi  of  the  mean  range  error  and  +1.43°  of  the  mean  azimuth  error. 
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DATA  METHODOLOGY  (SIMULATION  FLIGHT  TESTS) 


EDITING. 

Efficient  and  precise  repeatability  of  the  simulation  mode  of  testing  per- 
mitted editing  to  be  done  in  real  time  by  the  technique  of  selective  recording 
of  data.  The  selection  of  aircraft  parameters,  positions,  tracks,  and  numbers 
on  a programmed  basis  permitted  close  examination  of  particular  test  objectives. 
For  example,  to  investigate  beacon  code  garbling  due  to  reply  code  train  over- 
laps, i.e.,  multiple  targets  with  separations  (d)  equal  to  less  than  that 
defined  by  the  product  of  the  radar  propogation  velocity  (v)  and  the  time 
(t)  between  code  framing  pulses,  it  was  an  easy  matter  to  place  32  discrete 
aircraft  targets  in  trailing  groups  of  4 each  on  select  radials  with  target 
separations  varying  between  1/2  and  2 3/4  nmi  per  group  in  steps  of  1/4  nmi 
(figure  19).  Flight  plans  for  each  target  were  placed  in  storage  in  the  DPS 
prior  to  the  start  of  the  test.  The  test  targets  were  then  introduced  as 
programmed  from  the  RBS  scenario  tape.  The  thresholds  of  separation  at  which 
ungarbled  detection  and  reliable  tracking  occurred  were  readily  apparent  by 
observation  of  a radar  display.  Online  editing  was  accomplished  as  data  were 
collected.  Tests  were  readily  repeatable  to  ensure  high  confidence  in  results; 
in  addition,  test  designs  were  often  modified  online  to  benefit  from  just- 
prior  experience. 

REDUCTION. 

Online  monitoring  of  test  results,  reinforced  by  ease  of  playback,  coupled 
with  the  selective  reduction  programs  of  the  ARTS  III  UNISERVO  VIc  System, 
permitted  printed  output  of  test  results  on  a next-day  basis.  Existing  data 
reduction  and  analysis  (DR&A)  programs  (appendix  Bl)  provided  target  reports 
and  predicted  track  data  versus  sector  time.  In  addition,  motion  picture 
records  of  the  Radar  Display  were  processed,  and  by  selectable-speed  playback 
(both  forward  and  reverse),  areas  of  interest  could  be  quickly  identified. 
Page-size  prints  of  selected  frames  of  interest  were  available  for  detailed 
study. 

ANALYSIS. 

Tests  were  designed  to  produce  finite  and  unambiguous  results  by  the  near 
absolute  control  of  variables.  The  analysis  was  achieved  by  real  time  moni- 
toring of  ARTS  responses  to  specific  target  parameters.  Hard  copy  verifica- 
tion was  achieved  by  use  of  the  ARTS  III  9300  Processor  System  and  by  radar 
display  photos.  Simulation  focused  attention  on  specified  situations  in  a 
manner  wherein  results  become  almost  obvious. 
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FIGURE  19.  RBS  SCENARIO  000-MINIMUM  RANGE  SEPARATION  EVALUATION 


SCENARIO  000:  32  TARGETS,  4 ON  EACH  45*  OF  AZIMUTH,  VARIOUS  TRAIL 
SEPARATIONS  FROM  1/2  TO  2 1/4  NMI,  DISCRETE  BEACON  AND  RADAR 
(MTI),  INBOUND  FROM  40  NMI. 

SCENARIO  001;  AS  000,  EXCEPT  RADAR  ONLY. 
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RESULTS  (SIMULATION  FLIGHT  TESTS) 


GARBLING,  TRAIL  SEPARATION. 

Figure  19  depicts  RBS  scenarios  000  and  001  during  which  multiple  target 
trail  separations  varying  from  1/2  to  2 1/4  nmi  were  maintained  on  cardinal 
radials  (every  045°).  As  notea  in  radar  display  photograph  (figure  20), 
scenario  000,  which  shows  all  beacon  targets  with  stored  flight  plans,  garbling 
voided  all  attempts  to  establish  tracking  until  a separation  of  1 3/4  nmi  was 
reached.  Again,  however,  at  2 nmi,  mode  C garbling  and  false  idents  regularly 
occurred.  This  is  further  documented  in  figure  21,  UNISERVO  VIc  printout, 
which  shows  the  "report"  and  "track"  garble  condition  at  the  2-nmi  separation. 
Figure  22,  UNISERVO  VIc  printout,  is  included  to  contrast  radar-only  detection 
and  tracking  now  successfully  occurring  at  the  1/2-nmi  separation  level. 

GARBLING,  AZIMUTH  SEPARATION . 

Figure  23  depicts  RBS  scenarios  003  and  004  during  which  multiple  target  trail 
separations  again  were  varied  between  1/2  and  at  least  2 1/4  nmi.  This  test, 
however,  involved  targets  flown  abeam  to  the  ASR,  so  that  the  separation  would 
appear  in  the  azimuth  sense.  Again,  as  noted  on  radar  display  photograph 
(figure  24)  and  UNISERVO  VIc  printout  (figure  25),  garbling  voided  tracking 
until  a separation  of  1 3/4  nmi  was  achieved,  followed  again  by  mode  C garbling 
and  false  idents  at  the  2-nmi  limit.  Figure  26,  RBS  scenario  008  depicting 
beacon  targets  separated  by  slightly  greater  than  one  beamwidth  (5°),  reveals 
that  when  garbling  no  longer  occurred,  all  targets  were  successfully  detected 
as  shown  by  the  UNISERVO  VIc  printout  (figure  27).  Target  reports  on  suc- 
cessive sweeps  effectively  define  an  azimuth  dimension  of  one  ATCBI  beamwidth 
for  each  beacon  target.  Radar  display  photo  (figure  28)  again  reveals  that 
radar-only  targets  possess  this  limitation  to  a much  lesser  degree  (0.5  nmi), 
due  to  the  narrower  ASR  beamwidth  and  target  reply  duration. 

TRACK  SWAPPING. 


Disc. ete  beacon  codes  lose  their  fidelity  if  garbling  occurs  and  the  potential 
of  track  swaps  arises.  Figure  29  depicts  RBS  scenario  024.  Targets  in  flights 
of  two  or  three  aircraft  (first  when  discrete  beacon  was  used  and  then  radar- 
only)  were  flown  on  converging  courses  designed  to  cross  over  at  a point  20  nmi 
from  the  ASR  on  the  270°  radial.  Figure  30,  radar  display  photograph,  shows 
that  after  convergence  and  separation,  track  AA101  has  swapped  with  AA102, 
track  AA103  has  swapped  with  AA104,  and  BB104  has  swapped  with  BB105.  AA 
targets  are  discrete  beacon  radar-reinforced;  BB  targets  are  radar-only. 

Figure  31,  UNISERVO  VIc  printout,  shows  these  swaps,  i.e.,  reported  and  assigned 
beacon  codes  uncorrelated,  target  identifications  interchanged,  mode  C altitudes 
falsely  correlated,  etc. 

Since  these  data  were  taken,  the  program  has  been  reported  corrected,  which 
resulted  in  the  elimination  of  discrete  beacon  target  swaps  and  a marked 
improvement  for  radar-only  and  nondiscrete  beacon  targets.  The  results  of 
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FIGURE  22.  UNISERVO  Vic  PRINTOUT,  RADAR-ONLY  TRACKING,  RBS  SCENARIO  001 
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SCENARIO  00  3:  30  TARGETS  FLYING  30  nmi  ABEAM  OF 
THE  RADAR.  225  KNOTS.  HEADING  180*.  IN  T RAIL  SEPARATIONS 
VARYING  FROM  1/2  to  2 3/4  nmi,  IN  FORMATIONS  OF  3 TARGETS 
WITH  3 - nmi  SEPARATION  BETWEEN  GROUPS;  TARGETS  ARE 
DISCRETE  BEACON  WITH  RADAR  REINFORCEMENT.  TRACKS 
ARE  STARTED  FROM  STORED  FLIGHT  PLANS. 


SCENAPIO  004:  SAME  AS  003,  EXCEPT  RADAR  ONLY 


FIGURE  23.  RBS  SCENARIO  003/ 004- -MINIMUM  AZIMUTH  SEPARATION 
EVALUATION 


NAUTICAL  MILES 


SCENARIO  008  : 32  TARGETS  SEPARATED  BY  58  OF  AZIMUTH, 
INBOUND  FROM  40  N.  MILES,  DISCRETE  BEACON  AND  RADAR. 


SCENARIO  009  : AS  008,  EXCEPT  RADAR  ONLY  , RADAR  REPLY 
STRENGTH  VARIES  FROM  0 TO  31  DB.  WITH  INCREASING  AZIMUTH. 


FUGURE  26.  RBS  SCENARIO  008 — MINIMUM  AZIMUTH  SEPARATION 
EVALUATION 
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SCENARIO  024  : 10  TARGETS  CONVERGING  ALONG  2 OR  3 TRACKS 
AT  A POINT  20  N.  MILES  OUT  ON  THE  270°  RADIAL,  IN  COMBINATIONS 
OF  2 AND  3 TARGETS,  DISCRETE  BEACON  PLUS  RADAR  AND  RADAR  ONLY 
225  KNOT  AIRSPEED. 


FIGURE  29.  RBS  SCENARIO  024--TRACK  SWAP  EVALUATION,  CONVERGING 
TARGETS 
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FIGURE  31.  UNISERVO  Vic  PRINTOUT  SHOWING  TRACK  SWAPS,  RBS  SCENARIO  024 


swap  tests  with  the  improved  program  can  be  found  in  a MITRE  report  MTK-7300 
entitled  "Technical  Performance  of  the  ARTS  III  Tracking  Function." 

TRACKING  LIMITS  IN  TURNS. 

It  was  verified  that  as  long  as  the  round  reliability  was  unity,  track  reten- 
tion in  turns  was  solid,  even  during  abnormal  turn  rates.  Figure  32  depicts 
RBS  scenario  018  during  which  turn  rates  as  high  as  10° /second  were  tracked 
without  a track-coast  condition  developing;  target  airspeed  was  maintained  at 
180  knots.  The  displacement  of  the  target  in  relation  to  the  tracking  bin 
dimensions  is  the  prime  factor  for  track  retention  in  turns.  Figure  33,  RBS 
scenarios  022  and  023,  demonstrate  this.  A single  target  was  maintained  at  a 
constant  turn  rate  of  3°/second  and  accelerated  at  a constant  rate  (45  knots/ 
min)  from  an  initial  velocity  of  100  knots  to  a velocity  high  enough  to  cause 
loss  of  correlation  and  track  coast.  At  unity  round  reliability,  this  did 
not  occur  for  the  beacon  target  until  a velocity  in  excess  of  300  knots  was 
achieved.  Figure  34,  UNISERVO  VIc  printout,  shows  this  event.  Figure  35, 
another  UNISERVO  VIc  printout,  shows  a similar  radar-only  target  commencing 
to  fail  to  correlate  at  360  knots.  Any  degeneration  in  round  reliability,  as 
was  observed  in  live  testing,  seriously  degrades  this  performance. 

TRACKING  AT  LIMIT  SPEEDS. 

Figure  36  depicts  RBS  scenario  025  during  which  beacon  targets  were  acceler- 
ated to  determine  the  velocity  at  which  straight-line  tracking  would  fail  to 
correlate  and  a coast  condition  develop.  As  shown  in  figure  37,  a radar  dis- 
play photo,  this  occurred  at  700  knots. 

TRACKING  AT  CLOSE  RANGES. 

Maximum  position  errors  equivalent  to  the  height  of  the  target  were  experienced 
because  slant  range  rather  than  ground  range  was  processed  and  displayed. 
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SCENARIO  018  : 8 TARGETS,  180  KNOTS,  VARIOUS  TURN  PATES 
FROM  3 TO  I0a  / SEC.  , DISCRETE  BEACON  AND  RADAR.  T URN- 
CENTERS  ABOUT  A POINT  12  nmi  OUT  ON  THE  360“  RADIAL 


FIGURE  32.  RBS  SCENARIO  018 TARGETS  IN  VARIOUS  TURN  RATES 


NAUTICAL  MILES 


SCENARIO  Oil  : 1 TARGET,  ACCELERATING  FROM  100  TO  600 
KNOTS,  (45KNOTS/MIN.  ),  CLIMBING  FROM  2 TO  99.  9 K (1500 
FEET /MIN,  ).  EXECUTING  A SPIRAL  TURN  AT  A CONSTANT  RATE 
OF  V /SEC.  RADAR  AND  DISCRETE  BEACON. 


FIGURE  33.  RBS  SCENARIO  022/023  - TURNING  TRACK  EVALUATION 
3° /SECOND  TURN  RATES 


0-999  KNOTS 


20-999  KNOTS 


40-999  KNOTS 


b0-999  KNOTS 


80-999  KNOTS 


SCENARIO  025  : 20  TARGETS,  GROUPS  OF  5.  ACCELERATING  AT 
RATES  OF  45.  75,  120,  AND  22 5 KNOTS  MIN.  . INITIAL  VELOCITIES 
VARY  FROM  0 TO  80  KNOTS.  TERMINAL  VELOCITY  999  KNOTS. 


FIGURE  36.  RBS  SCENARIO  025— LIMIT  TRACK-SPEED  EVALUATION 


99/ '3/41  ? 9 97 


FIGURE  37.  RADAR  DISPLAY  PHOTO— RBS  SCENARIO  025  SHOWING  TRACK  COAST 
COMMENCING  AT  700  KNOTS 


CONCLUSIONS 


I 


AIRCRAFT  POSITION  ACCURACY. 

1.  The  accuracy*  to  which  ARTS  III**  can  report  aircraft  positions  is  +0.10 
nmi  in  range  and  +0.87°  in  azimuth.  Range-error  variability  is  primarily  a 
function  of  the  0.0625-nmi  range  resolution  of  the  ARTS  III  itself.  There  is 
no  correlation  between  the  magnitude  of  range  and  azimuth  reported  errors  and 
the  distance  of  the  target  from  the  radar. 

There  is  no  practical  difference  in  the  ARTS  III**  position-reporting  accuracy 
between  beacon-only  and  radar-reinforced  beacon  targets. 

There  is  no  practical  difference  in  ARTS  III**  position-reporting  accuracy 
between  radar-only  and  radar-reinforced  beacon  targets. 

2.  The  accuracy*  to  which  ARTS  III**  can  predict  successive  aircraft  posi- 
tions (track)  is  +0.40  nmi  in  range  and  +1.04°  in  azimuLh  when  the  aircraft  is 
in  tangential  flight.  The  corresponding  values  for  straight  flight  are 
+0.17  nmi  and  +0.51°. 

The  accuracy  of  the  target  reports  are  greater  than  those  of  the  predicted 
positions  for  both  straight  and  tangential  flight. 

3.  The  accuracy*  to  which  the  ARTS  III**  radar  display  can  display  aircraft 
position  is  +0.12  nmi  in  range  and  +1-04°  in  azimuth.  For  a calibrated  dis- 
play, this  error  variability  is  less  than  that  generated  in  trying  to  read 
position  from  the  radar  displays. 

4.  The  overall  accuracy*  to  which  ARTS  III**  can  report  and  display  aircraft 
positions  is  +0.16  nmi  in  range  and  +1.43°  in  azimuth.  Again,  these  errors 

are  less  than  those  generated  in  trying  to  read  position  from  the  radar  displays. 

AIRCRAFT  SEPARATION  ACCURACY. 

The  accuracy*  to  which  ARTS  III**  can  report  aircraft  separations  is  +0.163  nmi. 
Separation  error  variability  is  primarily  a function  of  the  0.0625-nmi  range 
and  0.088°  azimuth  resolution  of  the  ARTS  III. 

AIRCRAFT  RESOLUTION  ABILITY . 

The  ARTS  III  target  resolution  is  estimated  to  be  2.0  nmi  when  targets  are 
separated  in  range  from  the  radar  and  6°  when  separated  in  azimuth.  These  are 
90-percent  estimates,  i.e.,  90  percent  of  the  time,  when  both  targets  are 
separated  in  range  or  azimuth  by  at  least  the  amounts  indicated  above,  both 
aircraft  will  be  properly  identified. 
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MODE  C ALTITUDE  REPORTING  ACCURACY. 

The  accuracy*  to  which  ARTS  III**  can  report  mode  C altitudes  is  +96  feet, 
given  a standard  atmosphere.  Altitude  error  variability  is  primarily  a func- 
tion of  the  100  feet  altitude  resolution  of  the  aircraft  transponder. 

DISPLAYED  GROUNDSPEED  ACCURACY. 

the  accuracy*  to  which  ARTS  III**  can  predict  groundspeed  is  +32  knots.  Speed 
is  displayed  to  the  nearest  10  knots. 

DETECTING  AND  TRACKING  LIMITATIONS. 

Simulation  testing  reveals  that  a beacon  reply  occupies  a radar  area  of 
dimensions  2 nmi  in  range  and  one  ATCBI  antenna  beamwidth  (nominally  4.5°)  in 
azimuth.  The  azimuth  beamwidth,  in  linear  dimensions,  varies  directly  with 
range;  increasing  from  0.2  nmi  at  2,5-nmi  range,  to  2 nmi  at  the  25-nmi  range. 
Whenever  two  beacon  targets  are  within  this  common  area,  mode  A and/or  mode  C 
crosstalk  may  occur,  resulting  in  garbled  codes  and/or  altitudes,  and  making 
track  swaps  possible.  Whenever  three  beacon  replies  emanate  from  this  common 
area,  track  swaps  are  virtually  certain  to  occur.  The  airspace  requirements 
of  a radar-only  target  are  defined  primarily  by  the  ASR  antenna  beamwidth 
(nominally  1.5°).  Targets  as  close  as  0.2  nmi  can  be  tracked  if  their  azimuth 
separation  is  at  least  one  beamwidth.  Whenever  two  or  more  radar-only  targets 
possess  less  than  this  separation,  track  validity  is  lost. 

Track  retention  during  turning  is  primarily  dependent  on  the  maintenance  of 
unity  round  reliability,  a situation  not  normally  achievable  in  the  live  envi- 
ronment. In  simulation,  where  unity  round  reliabilty  was  maintained,  track 
retention  was  shown  to  be  almost  independent  of  turn  rate  under  normal  maneu- 
vering conditions;  i.e.,  as  long  as  flightpath  accelerations  do  not  result 
in  displacements  that  exceed  tracking  bin  dimensions.  In  live  testing,  where 
round  reliability  in  turns  regularly  drops  to  0.8  to  0.9,  displayed  track 
position  and  airspeed  errors  are  greatly  magnified.  Controller  entries  to 
reposition  tracks  are  often  required  to  insure  track  retention. 

The  ARTS  III  RBTL  performance  with  nonbeacon-equipped  aircraft  was  limited  b) 
the  lower  round  reliability  (observed  to  average  approximately  0.5);  this  was 
attributed  to  variations  in  the  signal-to-noise  ratio  of  the  radar  return  as 
range,  track,  aircraft  attitude,  and  meteorological  conditions  changed.  The 
detection  probability  of  beacon-equipped  aircraft  similarly  was  only  slightly 
enhanced  by  radar  reinforcement. 

* 99.9  percent  of  the  errors  can  be  expected  to  fall  within  the  stated 

limits  about  the  mean  error  with  a confidence  of  90  percent. 

**  ARTS  III  (RBTL) /ATCBI-3 
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APPENDIX  A 

NAFEC  TEST  SUPPORT  FACILITIES 


Figure  Page 

A-la  NAFEC  Facilities  Map  A-J 

A-lb  NAFEC  Facilities  Map  Legend  A-2 

A-2  Plan  View  of  the  Terminal  Automation  Test  Facility  A-3 

A- 3 Terminal  Facility  for  Automation  and  Surveillance  A-4 

Testing  (TFAST) 

A--t  Typical  NAFEC  Test  Aircraft  A- 5 

A- 5 Instrumentation,  Typical  NAFEC  Test  Aircraft  A-6 

A-6  Phototheodolite  Instrument  A- 7 

A- 7 Phototheodolite  Data  Flow  Diagram  A-8 

A- 8 Extended  Area  Instrumentation  Radar  (EAIR)  A- 9 

A- 9 EAIR  Data  Flow  Diagram  A-10 

A-10  Range  Control  Central  Facility  A-ll 

A-ll  Data  Preparation  Facility  A-I2 

A-12  Data  Processing  Facility  A-13 

A-13  Photographic  Instrumentation  Processing  Area  A-14 


Pholoiheodolite  Tower  P36  (Building  187)  Test  Conlrol  Facility  (NAFLl  Radai) 

Antenna  Test  Range  (Building  192)  Test  Control  Coordination  Facility 

Pholoiheodolite  Tower  P8  (Building  184)  No.  300  Switching  System  Facility 

Fire  Fighting  Test  Facility  Compulation  Facility 

Upper  Alt  Facdity  (Building  193)  32  ^ho.'“Bra^‘.C  Seryices  Facili,y  (BuildinB  48> 


System  integration  Test  Environment  (SITE)  Facility  76-6-A-lb 


ARTS  m DISPLAY  LAB  ( 1600  SO.  FT.) 
COMMUNICATIONS  EQUIPMENT  ROOM  (845  SQ.  FT.) 
SIMULATION  PILOT'S  DISPLAY  ROOM  (255  SQ.  FT  ) 
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FIGURE  A- 12 . DATA  PROCESSING  FACILITY 
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APPENDIX  B1 

UNIVAC  EXTRACTOR  TAPE  PROGRAM 


Figure  B-l  shows  a sample  of  the  printout  produced  by  this  ARTS  III  program. 

The  program  prints  out  the  target  reported  and  predicted  positions  that  occur 
within  each  sector  of  azimuth.  There  are  32  sectors  per  scan;  hence,  each 

sector  comprises  11.25°  or  128  azimuth  change  pulses  (ACP's).  On  the  extreme 

left  of  the  printout  is  the  sector  time.  Looking  at  the  line  marked  (a)  in 
figure  B-l,  it  will  be  noted  that  this  sector  began  at  10:35:57.773  hours. 

On  the  line  below,  marked  "1  TARGET  RPT",  it  will  be  noted  that  three  beacon 

target  reports  occurred  during  this  sector.  Looking  at  the  first,  or  left- 

most target  report,  the  character  group  1768W  will  be  noted  to  the  extreme 
left.  The  1768  is  the  azimuth  of  the  target  in  ACP's.  This  corresponds  to  a 
bearing  of  155.39°.  The  symbol  "W"  is  one  of  two  symbols  which  indicate  target 
strength.  This  is  a function  of  the  number  of  hits  in  the  beacon  sliding  win- 
dow, which  is  used  for  the  determination  and  positioning  of  the  beacon  target. 
The  symbols  are  "W"  for  weak  targets  and  "S"  for  strong  targets.  A weak  tar- 
get is  indicated  by  up  to  seven  or  eight  hits  in  the  sliding  window  (depending 
on  mode  interlace);  a strong  target  report  is  indicated  by  eight,  nine, 
or  more  hits  in  the  sliding  window. 

The  next  group  of  characters  reading  to  the  right  is  03.88.  This  is  the 
range  of  the  target  in  nautical  miles.  Following  this  is  the  group  of  charac- 
ters 0176-3.  The  0176  is  the  mode  3/A  beacon  identity  (ID)  code.  The  "3"  is 
the  mode  3/A  validity  figure.  The  validity  figure  is  a figure  of  merit  con- 
cerning the  accuracy  of  the  received  beacon  ID  code  (0176).  The  number  "3" 
is  the  highest  possible  value  for  this  mode  3/A  validity  figure. 

Following  the  mode  3/A  character  group,  the  characters  "031-2"  will  be  noted. 
The  "031"  stands  for  the  mode  C beacon  altitude  in  hundreds  of  feet  (i.e., 

3,100  feet),  and  the  "2"  after  it  is  the  mode  C validity  figure,  which,  like 
the  mode  3/A  validity  figure,  is  a figure  of  merit  concerning  the  accuracy 
of  the  received  beacon  mode  C altitude  code.  As  in  the  case  of  the  mode  3/A 
validity  figure,  the  number  "3"  is  the  highest  possible  value  for  the  mode  C 
validity  figure. 


The  final  character  in  the  target  report  is  a numeral  ranging  from  0 to  7. 
This  is  the  radar  quality  (RQ)  number,  which  is  an  indication  of  the  number 
of  hits  in  the  radar  sliding  window.  For  the  target  report  under  discussion, 
it  will  be  noted  that  this  character  is  a "0".  This  indicates  no  radar  re- 
turns at  all  from  the  target;  hence,  this  is  a beacon-only  target.  An  RQ 
indication  of  1 is  the  weakest  radar  target,  indicating  eight  or  less  hits  in 
the  radar  sliding  window.  An  RQ  indication  of  7 is  the  strongest  radar  tar- 
get, indicating  19  or  more  hits  in  the  radar  sliding  window. 
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The  second  target  report  in  this  sector  is  a strong  target  at  1,820  ACP's  in 
azimuth,  3.25  nmi  in  range,  with  beacon  mode  3/A  ID  code  of  0177,  mode  3/A 
validity  of  3,  at  an  altitude  of  3,500  feet  with  a mode  C validity  of  3,  and 
a radar  quality  of  4.  The  third  target  report  in  the  sector  is  a strong 
target  at  1,806  ACP's  azimuth,  3.88  nmi  in  range,  with  beacon  ID  of  6176  and 
at  0 feet  altitude.  It  will  be  noted  that  the  validity  numbers  for  both  beacon 
ID  and  altitude  are  each  only  1,  indicating  poor  accuracy  of  beacon  ID  and 
altitude  decodings.  Since  this  target  is  at  the  same  range  and  almost  the 
same  azimuth  as  the  first  target  report,  and  has  a similar  beacon  ID  code 
(6176  versus  0176),  the  third  target  report  may  represent  a misdecoding  or 
garble  of  the  0176  code. 

Looking  at  line  b,  we  see  that  for  the  sector  beginning  at  10:35:58.070, 
there  were  three  target  reports.  The  first  two  groups  of  characters  are  simi- 
lar to  those  of  the  line  above,  except  there  is  no  symbol  for  beacon  strength 
in  the  first  character  group.  The  first  target  report  is  seen  to  be  at  2,081 
ACP's  in  azimuth  and  57.88  nmi  in  range,  while  the  second  target  is  at  an 
azimuth  of  2,080  ACP's  and  a range  of  58.25  nmi.  The  third  character  group, 
which  had  previously  indicated  the  mode  3/A  beacon  ID  code,  now  indicates 
"RADAR".  This  means  that  the  target  report  consisted  of  primary  radar  returns 
only;  there  were  no  beacon  returns  at  all  from  the  target.  The  lone  digit 
on  the  right,  as  before,  represents  the  radar  quality,  or  RQ,  which,  for  both 
radar-only  targets  in  this  sector,  is  a 7,  or  highest  quality  radar  return. 

The  right-most  target  in  line  b is  a strong  beacon-only  target  at  2,080 
ACP's  azimuth,  59.06  nmi  range,  with  beacon  ID  code  of  0000  with  mode  3/A 
validity  of  3.  The  fourth  or  altitude  group  for  this  target  is  0000-0,  which, 
since  the  mode  C validity  is  0,  indicates  no  mode  C altitude  transmission 
from  the  transponder.  Line  c shows  a predicted  target  position  occurring 
at  10:35:58.474  hours.  This  is  a predicted  position  on  aircraft  N376.  The 
character  "R"  to  the  right  of  the  aircraft  ID  group  indicates  that  the  pre- 
diction was  based  upon  a radar  or  beacon  target  report,  rather  than  an  extra- 
polated position  or  "coast"  mode,  for  which  the  symbols  "CST"  would  be  dis- 
played. The  next  group  of  characters,  031,  indicates  the  aircraft  altitude 
in  hundreds  of  feet — i.e.,  3,100  feet.  Following  this  are  two  identical  groups 
of  characters,  0176,  indicating  the  mode  3/A  beacon  identity  code.  The  next 
group  of  characters,  1806,  is  the  azimuth  of  the  track  in  ACP's.  The  next 
three  groups  of  characters  represent  the  range  (3.63  nmi),  and  the  x and  y 
components  of  the  range.  The  next  group  of  characters  (F47)  represents  the 
track  firmness,  in  octal  notation.  The  track  firmness  is  a figure  of  merit 
concerning  the  quality  of  the  track.  The  numeral  "47"  in  octal  notation  corres- 
ponds to  39  in  decimal  notation,  which  is  the  highest  value  of  track  firmness. 
The  next  character  group  "VEL  22"  indicates  a track  velocity  of  220  knots. 
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ARTS  III  RBTL  UNPACK  PROGRAM 

i 

Figure  B-2  shows  a sample  of  the  printout  produced  by  this  program.  The  selec- 
ted beacon  ID's  in  this  case  were  0176  and  0177.  Lines  a and  c correspond 
to  the  reported  and  predicted  target  position  messages  shown  in  lines  a 
and  c,  respectively,  of  the  extractor  tape  listing  sample  (figure  B-l) . 

It  will  be  seen  that  for  the  two  target  reports  marked  "a";  the  time,  mode 
3/A  beacon  ID's,  mode  C altitudes,  and  RQ's  are  the  same  as  those  shown  for 
the  corresponding  target  reports  on  line  a of  figure  B-l.  The  azimuth 
readings  have  been  changed  from  ACP's  to  degrees,  and  the  validity  and  beacon 
strength  indications  are  shown  as  the  three  right-most  digits  under  the  column 

heading  "REFACW  FLAGS''.  The  most  significant  digit  in  this  group  under  the 

"A"  of  REFACW,  is  the  mode  3/A  validity  number,  which,  for  both  targets,  is  3. 
The  second  digit,  under  the  "C"  of  REFACW,  is  the  mode  C validity  number. 

This  is  2 for  the  0176  target  and  3 for  the  0177  target,  corresponding  to  what 
was  displayed  in  figure  B-l.  Finally,  the  W and  S beacon  strength  indication 
characters  of  figure  B-l  have  been  given  the  numerical  codings  of  0 and  1, 

respectively,  and  are  so  shown  under  the  "W"  of  REFACW. 

In  the  case  of  the  predicted  position  shown  in  line  c,  it  will  again  be  noted 
that  the  items  correspond  to  the  values  shown  in  line  c of  figure  B-l,  except 
that  the  azimuth  is  displayed  in  degrees,  and  track  firmness  is  indicated  in 
decimal  notation  rather  than  octal.  The  aircraft  groundspeed  in  knots, 
together  with  its  x and  y components,  are  also  provided. 


FIGURI'  B-2  RBTL  UNPACK  PVKJrAM 
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APPENDIX  Cl 

SIMPLE  LINEAR  REGRESSION  AND  CORRELATION  ANALYSES 

PURPOSE. 

1 

The  purpose  of  this  analysis  is  to  quantify  the  relationship  between  some 
independent  variable,  x,  and  some  dependent  variable,  y,  based  on  the  uni- 
verse regression  line,  y=ot+Bx,  where  a is  the  y intercept  and  3 is  the  slope 
of  the  line. 

PROCEDURES. 

Using  the  method  of  least  squares,  the  coefficients  a and  B of  the  regres- 
sion line  can  be  estimated  from  a sample  of  n pairs  of  x and  y data  using 
a and  b where  „ 

= (Zy)  (Zx2)  - (Ex)  (Exy). 

n(Zx2)  - (Zx)2 


i 


and 

b = njZxy)  - (Zx)  (Zy) 

n(Zx2)  - (Zx)2 


The  estimated  regression  line  is  therefore  y=a+bx.  A measure  of  the  error 
involved  in  using  the  regression  value  to  estimate  y is  commonly  referred 
to  as  the  standard  error  of  estimate,  se. 


where 


Zy2  - a (Zy)  - b(Zxy) 
n - 2 


The  prediction  limits  or  limits  for 
for  x=x0  as  follows: 

(a  + bx0)  + 


individual 
ca/2  se 


values  of  y can  be  estimated 

1 + 1 + n(*  o -J?  ) 2 
n n(Zx^)  - (Zx)^ 


where  ta/2  is  obtained  from  the  t-distr ibution  tables  for  n-2  degrees  of 
freedom.  These  limits  will  contain  y at  x=xQ  with  a probability  of  1-a. 
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ASSUMPTIONS . 

This  analysis  assumes  that  for  each  value  of  x,  y lies  on  the  universe 
regression  line.  It  also  assumes  that  the  values  of  y for  a given  value  of 
x are  normally  distributed  and  that  the  standard  deviation  of  these  values 
is  the  same  for  all  x values. 

SIMPLE  CORRELATION  ANALYSIS. 

PURPOSE . To  quantify  the  degree  of  linear  association  between  two  variables 
x and  y,  based  on  the  coefficient  of  correlation. 

PROCEDURES . An  estimate  of  the  population  coefficient  of  correlation  is 
calculated  as  follows: 


r = n(Zxy)  - (£x)(by) 

Vn  (Zx2)  - (lx)2  Vn(Lyz)  - (j  y)2 


More  explicitly  known  as  the  Pearson  Product- Moment  Coefficient  of  Corre 
lation , r varies  between  -1.0  and  +1.0.  Perfect  correlation  is  indicated 
by  r = +1.0.  When  r = 0,  no  correlation  exists. 

ASSUMPTIONS . This  analysis  assumes  that  the  sample  regression  line  and 
the  standard  error  of  estimate  are  derived  by  the  method  of  least  squares. 
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F-TEST  FOR  DIFFERENCES  BETWEEN  VARIANCES 


PURPOSE. 

The  purpose  of  this  test  is  to  determine  whether  the  variability  of  one  set 
of  data  is  significantly  different  from  the  variability  of  another  set. 

PROCEDURE . 

2 2 

The  variances,  Sp  and  S 2 , of  each  of  the  two  sets  of  data  are  computed 
according  to  the  formula 

g 2 __  2 (xij  - xi) 

1 “ n^TT  ; where 

2 

S^  = variance  of  the  ith  set  of  data 

xpj  = value  of  the  jth  observation  in  the  ith  set 

of  data 


and 


xp  = mean  of  the  ith  set  of  data 

np  = number  of  observations  in  the  ith  set  of  data 


The  null  hypothesis  is  that  the  variances  of  both  sets  of  data  are  equal, 
i.e.  , 

H0;  Si2  = S22' 

The  ratio  of  the  two  variances,  S^/So2  ps  then  formed.  The  larger  of 
the  two  variances  is  placed  in  the  numerator  of  this  ratio.  This  ratio  is 
an  F-distribution  characterized  by  two  parameters  vl  and  .2.  vl , the 
degrees  of  freedom  associated  with  the  set  of  data  with  the  larger  variance, 
Sp  , is  equal  to  np-1.  Similarly  v2  = 112 -1. 

The  computed  value  of  F obtained  above  is  compared  with  the  tabular  value 
of  F for  vp  and  V2  degrees  of  freedom,  at  an  a/2  level  of  significance 
since  this  is  a two-sided  test.  If  the  computed  value  of  F is  smaller 
than  the  corresponding  tabular  value  then  the  null  hypothesis  that  the 
variances  of  the  two  sets  of  data  are  equal  is  not  contradicted  by  the  data. 


ASSUMPTIONS . 

This  analysis  assumes  that  both  sets  of  data  are  independent  and  normally 
distributed . 
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BARTLETT'S  TEST  FOR  HOMOGENEITY  OF  VARIANCES 


PURPOSE. 

The  purpose  of  Bartlett's  test  for  homogeneity  of  variances  is  to  test 
whether  the  estimated  variances,  Sj^,,  of  each  of  g sets  of  data  could  have 
all  come  from  the  same  universe  or  could  be  represented  by  a common 
homogeneous  parameter,  a^. 

PROCEDURE. 


The  population  consists  of  g samples,  each  sample  of  which  consists  of  n^ 
members.  Each  sample  has  an  associated  computed  variance,  S^“,  and  an 
associated  number  of  degrees  of  freedom,  v^n^-l.  The  null  hypothesis  is 
that  the  variances  of  each  of  the  g samples  are  equal  i.e.,  Ho;  S]^  = 


2 

Using  the  and  vj_  of  each  of  the  g samples,  the  null  hypothesis  is 
tested  by  means  of  the  following  mathematical  models: 


M = 2.3026  jv  log1Q  ( Evi  S^/v 


= 1 + 


3 (g  - 1) 


where 


The  ratio  M/C  can  be  approximated  by  a chi-square  distribution  with  g-1 
degrees  of  freedom.  Therefore,  the  computed  M/C  ratio  obtained  above  is 
compared  with  the  tabular  value  of  chi-square  for  g-1  degrees  of  freedom, 
at  a a/2  level  of  significance,  since  this  is  a two-sided  test.  If  the 
M/C  ratio  does  not  exceed  the  tabular  value  of  chi-square,  then  the  null 
hypothesis  that  the  variances  of  the  samples  tested  are  from  the  same 
population  is  not  contradicted  by  the  data. 

ASSUMPTIONS ■ 

This  analysis  assumes  that  the  random  variations  within  each  of  the  g samp- 
are  normally  distributed  and  that  the  estimates  of  variance,  S-^2,  from 
each  of  the  g samples  are  independent.  This  test  also  assumes  a minimum 
of  five  members  per  sample,  i. 
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ONE-WAY  ANALYSIS  OF  VARIANCES 


PURPOSE. 

The  purpose  of  the  one-way  analysis  of  variance  is  to  determine  whether 
any  real  differences  exist  between  the  k means  of  a population  n. 

PROCEDURE. 

The  population,  n,  is  divided  into  k independent  random  subsets  of  sizes 
ni,  n2,  - - - n^,  where  ni  is  the  number  of  items  in  the  ith  subset,  xi  is 
the  estimated  mean  of  the  ith  subset,  and  sf  is  the  estimated  standard 
deviation  of  the  ith  subset. 

The  null  hypothesis  is  that  the  means  of  each  subset  (treatment  means)  are 
equal,  i.e.,  -H0;  ui  = u2  = = uk. 

Tne  null  hypothesis  is  tested  by  the  F-statistic,  which  is  constructed  by 
the  following  mathematical  model: 


F 


where 


where 


sm2  = 1 <*i  “ x) 

i=l 

_ — - 


k 

^ *1  ni  and 

“k 

Z n^ 
i=l 


Sp  = Z (ni  - 1)  Si2 
i-1 
k 

Z ni  - k 
i=l 


The  computed  value  of  F above  is  compared  with  the  tabular  value  of  F for 
vi=k-l  and  v2=n-k  degrees  of  freedom  at  an  a level  of  significance.  If 
the  computed  value  of  F is  smaller  than  the  corresponding  tabular  value, 
then  the  null  hypothesis  that  the  means  of  the  k subsets  are  equal  is  not 
contradicted  by  the  data.  It  should  be  noted  that  the  analysis  of  variance 
test  is  always  a "one-sided"  test. 


ASSUMPTIONS. 

This  analysis  assumes  that  the  observations  are  independent  and  normally 
distributed,  with  a common  variance  and  a mean,  ui,  for  each  of  the  k 
subsets.  It  is  further  assumed  that  the  errors  about  each  subset  mean, 
ui,  are  independent  and  normally  distributed  with  a zero  mean  and  a 
constant  variance,  o2. 
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ASPIN-WELCH  TEST  FOR  THE  DIFFERENCE  BETWEEN  TWO  MEANS  WITH 
UNKNOWN  AND  UNEQUAL  SAMPLE  VARIANCES  (Reference  12,  p.505) 


PURPOSE. 

This  test  is  to  determine  if  the  means  from  two  samples  are  from  the  same 
underlying  population,  given  that  the  population  variances  are  unknown  and 
may  be  unequal. 

PROCEDURE. 

The  two  populations  have  nq  and  n2  members,  sample  means  of  xp  and  x2, 
true  means  of  uq  and  u2,  and  sample  variances  of  si2  and  s2  respectively. 
The  null  hypothesis  is  that  the  difference  between  the  true  means  is  equal 
to  zero,  i.e.-  H0;  uq-u2=0. 

The  null  hypothesis  is  tested  by  means  of  the  t-statistic 
t = (xj_  - x2)  ~ (uq  ~ P2) 


and  treating  the  above  as  if  it  had  a t-distribution  with  degrees  of  free- 
dom, v , given  by 

1 

v - ^ —2 


nl  " 1 


where 


c = ' ri| 


nl  + s2  n2 


The  calculated  values  of  t andv  obtained  above  are  entered  into  a standard 
t-table  for  \>  degrees  of  freedom  and  Y / 2 level  of  significance  (since  this 
is  a two-sided  test).  If  the  calculated  value  of  t is  less  than  the 
corresponding  tabular  value,  then  the  null  hypothesis  that  the  differences 
of  the  true  means  is  zero  is  not  contradicted  by  the  data. 
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STATISTICAL  ANALYSIS  OF  ARTS  III  (RBTL)  TARGET 
REPORT  RANGE  AND  AZIMUTH  ERRORS  (EAIR  TRACKING) 
by  John  J.  Wojcie_h,  ANA-140  (May  1975) 


INTRODUCTION. 

Four  days  of  flights  were  conducted  at  NAFEC  to  determine  the  range  and  azimuth 
accuracy  of  the  ARTS  III  RBTL/ASR-5  system  using  the  Extended  Area  Instrumenta- 
tion Radar  (EAIR).  As  shown  in  figure  Dl-1,  the  horizontal  flight  profiles 
were  generally  the  same  for  each  of  the  flights,  except  that  a different 
quadrant  was  flown  each  day.  The  vertical  flight  profiles  shown  in  figure  Dl-2 
do  show  some  variation  from  day  to  day.  Each  flight  consisted  of  two  to  four 
runs  inbound  and  outbound  using  beacon  code  "0722."  NAFEC  aircraft  N103  was 
flown  on  November  18  and  25,  1974,  and  N377  flew  on  November  27  and  December  4, 
1974.  EAIR  range  and  azimuth  accuracies  are  listed  to  be  0.01  nmi  (60  feet) 
and  0.01°,  respectively.  Specific  design  and  test  condition  data  concerning 
the  ARTS  III/ASR-5  system  including  the  RBTL  computer  program  will  be  included 
in  the  final  report  for  activity  142-177-040. 


ERROR  SUMMARIES . 

A validated  program  developed  by  the  Data  Processing  Branch,  ANA-550,  was 
used  to  merge  the  EAIR  data  tapes  with  those  of  the  ARTS  III.  A sample  copy 
of  this  data  is  shown  in  figure  9.  Specifically,  EAIR  range  and  azimuth 
values,  translated  and  rotated  to  those  of  the  ASR-5  (latitude:  39°26'08.0", 
longitude:  74°35'02.1",  height:  100  feet,  rotation:  -9)  were  subtracted  from 
"time-correlated"  ARTS  III  target  reports.  The  program  was  designed  to  merge 
only  good  quality  (mode  3/A  and  mode  C validity  = 3;  strong  target 
indicator  = 1)  beacon  reports.  This  was  done  for  several  reasons,  the  primary 
one  being  that  over  90  percent  of  the  beacon  data  were  of  good  quality.  There 
was  no  filtering  initially  applied  with  respect  to  the  quality  of  the  radar 
reports,  since  this  did  vary  significantly  during  the  testing.  Finally, 
multiple  targets  within  the  same  scan  caused  by  reflections  or  "ring-around" 
were  also  not  processed. 

ERROR  DATA  GROUPING.  In  an  effort  to  maintain  homogenous  samples,  the  range 
and  azimuth  errors  were  grouped  by  azimuth  (290°-310°,  220°-240°,  0°-20°, 
and  120°-140°),  flight  direction  (inbound,  outbound),  and  radar  condition 
(merged,  beacon-only).  Merged  reports  are  a combination  of  beacon  reports 
and  radar  reports,  where  the  radar  reports  are  based  on  19  or  more  hits. 
Beacon-only  reports  include  no  radar  reports.  Data  points  or  errors  associated 
with  each  of  these  groups  were  randomly  selected.  Where  less  than  40  points 
were  listed,  all  the  data  were  considered  for  analysis. 

ERROR  DATA  SCREENING  AND  DISTRIBUTION  APPROXIMATION . The  errors  within  each 
analysis  group  were  checked  for  reasonableness.  In  a few  instances, 
statistical  criteria  were  applied  to  eliminate  possible  "outliers."  All 


points  selected  were  approved  and  the  error  distributions  were  declared  to  be 
homogeneous.  Finally,  a check  was  made  of  the  distributions  of  the  errors  in 
each  group.  As  indicated  by  the  histograms  in  appendix  DlA,  these  distribu- 
tions all  showed  to  be  generally  normal,  or  Gaussian. 

ERROR  1’ARAMETER  ESTIMATION.  The  following  analyses  were  performed  using 
Wang  700C  computer  programs.  Each  of  the  error  data  groups  was  analyzed  in 
accordance  with  the  procedures  contained  in  appendix  Cl  to  determine  if  there 
was  a linear  relationship  between  the  values  of  the  error  and  the  distance  of 
the  aircraft  from/to  the  ASR-5.  The  results  of  the  range  error  analyses 
. indicated  little  or  no  correlation  with  radar  range  for  the  "merged"  data 

and  fair  to  good  correlation  for  the  "beacon-only"  data.  In  the  case  of  the 
azimuth  errors,  little  or  no  correlation  was  shown  with  range  for  both  radar 
conditions.  On  the  basis  of  these  results,  the  error  values  were  lumped 
together,  and  maximum  likelihood  estimates  were  computed. 

RANGE  ERROR  ESTIMATES.  Estimates  of  the  mean  and  standard  deviation  (vari- 
ability) of  the  range  errors  for  each  of  the  error  data  groups  are  shown  in 
table  Dl-1.  Minus  values  of  mean  range  error  indicate  that  the  ARTS  III 
reported  data  were,  on  the  average,  less  than  that  recorded  by  EAIR. 

Conversely,  plus  mean  values  indicate  that  ARTS  III  measurements  were,  on  the 
average,  greater  than  those  of  EAIR.  In  order  to  obtain  the  best  overall 
estimates  of  range  accuracy,  tests  of  homogeneity  were  performed;  first 
using  the  standard  deviation  estimates  for  each  flight,  and  then,  where 
possible,  tests  were  performed  on  the  mean  estimates.  Where  allowable, 
combined  estimates  of  both  standard  deviations  and  means  were  computed  as 
shown  in  table  Dl-2. 

a.  Pooled  Standard  Deviations.  The  statistical  test  procedures 
shown  in  appendix  C3  were  applied  to  determine  if  the  standard  deviations 
estimated  for  each  flight  could  be  assumed  to  be  from  the  same  normal  distri- 
bution. Classification  was  by  flight  direction  and  radar  condition.  Except 
for  the  "outbound,  merged"  data,  the  standard  deviations  within  each 
classification  did  not  vary  significantly,  and  therefore,  combined  or  pooled 
standard  deviations  were  computed  using  all  4 days.  For  the  "outbound,  merged" 
case,  November  25,  27,  and  December  4,  1974,  results  were  combined. 

The  November  18  data  were  not  included,  since  they  were  considered  to  be 
significantly  different  from  that  of  the  other  3 days. 

i 

b.  Grand  Means . The  statistical  test  procedures  shown  in  appendix  C4 
were  used  to  determine  if  the  means  estimated  for  each  flight  could  be  assumed 

* to  be  from  the  same  normal  distribution.  Classification  was  again  by  flight 

direction  and  radar  condition.  For  both  radar  conditions,  the  "inbound"  means 
for  November  25,  27,  and  December  4,  1974,  were  not  significantly  different 
and  therefore  were  combined  and  a grand  mean  was  computed  using  this  data. 

The  November  18  data  were  significantly  different  from  those  of  the  other 
3 days  for  both  radar  conditions.  In  the  case  of  the  outbound  data,  a high 
degree  of  variability  was  indicated  for  both  radar  conditions,  and  therefore, 
these  data  were  not  combined. 
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TABLE  Dl-1.  RA”OL  ERROR  ESTIMATES  (nmi) 


TABLE  Dl-2.  COMBINED  RANGE  ERROR  ESTIMATES1 


Flight 

Direction 

Radar 

Condition 

Pooled 
Standard 
Deviation 
Sp  (nmi) 

Grand 
^Mean 
X (nmi) 

Xolerajorp.  Limits 

X - 3Sp 
(nmi ) 

X + 3Sp 
(nmi) 

X + 3Sp 
(6Sp) (nmi) 

Inbound 

Beacon-Only 

0.0272 

-0.0563 

-0.137 

0.025 

0.162 

(164) 

(-340) 

(-832) 

(152) 

(934) 

N=lll 

N=81 

Merged 

• 0232 

- .094 3 

- .163 

.025 

.138 

. 

(140) 

(-571) 

(-990) 

(-152) 

(839) 

N=147 

N=112 

Outbound 

Beacon-Only 

. 0282 

Note4 

.168 

(170) 

(1021) 

N=124 

Merged 

.0163 

Note4 

.096 

(97) 

(583) 

N=107 

lARTS  III  target  reports  minus  EAIR  from  10  to  40  nmi  from  the  ASR-5. 

The  procedures  used  to  determine  homogeneity  of  standard  deviations  and  means  are 
given  in  appendixes  C3  and  C4,  respectively,  (level  of  significance  = .01). 


2A11  four  days  were  combined. 

^November  25,  27,  and  December  4,  1974,  data  were  combined  (November  18  mean 
was  significantly  different). 

^Significant  differences  precluded  combining  this  data. 

Legend:  ( ) = Feet 

N = Combined  sample  size 

Formulas 

/ j 2 2 

SP  = 1/  (N1'1)S1  + (n2_1)S2  +....+<Nk-l)Sk  j 

" Nx  + N2  + ;:;+Nk  -k 

X = Nj  Xj  + N2X22  + •■•+NkXk 

% + . . • • -+Nk 


. 
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c.  Tolerance  Limits.  Using  the  grand  means  and  pooled  standard  devia- 
tions, 3-sigma  limits  were  calculated  as  shown  in  table  Dl-2.  In  addition,  the 
range  of  errors  based  on  the  3-sigma  limits  is  given.  The  3-sigma  limits 
indicate  that  at  least  99  percent  of  the  individual  range  errors  associated 
with  the  classifications  listed,  can  be  expected  to  be  within  these  limits. 

The  difference  between  the  3-sigma  limits  infers  that  at  least  99  percent 
of  the  range  errors  can  be  expected  to  be  within  the  range  derived. 

AZIMUTH  ERROR  ESTIMATES.  Estimates  of  the  mean  and  standard  deviation 
(variability)  of  the  azimuth  errors  for  each  of  the  error  data  groups  is  shown 
in  table  Dl-3.  Plus  mean  values  indicate  that  the  ARTS  III  reported  data  were, 
on  the  average,  greater  (more  clockwise)  than  that  recorded  by  EAIR.  Again, 
tests  of  homogeneity  were  performed  tc  determine  if  the  test  results  could 
be  combined  to  provide  a more  precise  estimate  of  azimuth  accuracy.  The 
combined  estimates  are  shown  in  table  Dl-4. 

a.  Pooled  Standard  Deviations.  The  statistical  test  procedures  shown 
in  appendix  C3  were  again  applied  to  determine  if  the  standard  deviations 
within  each  "flight  direction,  radar  condition"  test  classification  could  be 
combined.  Except  for  "outbound,  beacon-only"  data,  the  standard  deviations 
from  all  4 days  showed  to  be  homogenous.  A combined  estimate  was  obtained 
for  the  "outbound,  beacon-only"  case,  however,  using  the  results  from 
November  18,  27,  and  December  4,  1974.  The  November  25,  1974,  standard 
deviation  was  significantly  different  from  those  of  the  other  3 days. 

b.  Grand  Means . The  procedures  shown  in  appendix  C4  were  used  again 

to  statistically  test  for  significant  differences  between  the  means  within  each 
test  classification.  For  each  classification,  the  November  18,  25,  and 
December  4 means  can  be  considered  from  the  same  normal  distribution.  The 
November  27  means  were  significantly  different  from  those  of  the  other 
3 days. 

c.  Tolerance  Limits.  The  3-sigma  limits  and  the  differences  between 
these  limits  are  also  given  in  table  Dl-4. 

COMPARISON  TESTING. 

These  analyses  were  performed  using  Wang  700C  computer  programs.  Based  on  the 
statistical  test  procedures  described  in  appendices  C2  and  DIB,  the  range 
and  azimuth  error  pooled  standard  deviations  and  grand  means  given  in  tables 
Dl-2  and  Dl-4,  respectively,  were  compared  to  determine  the  effects  of  flight 
direction  and  radar  condition.  The  results  of  these  analyses  are  shown  in 
table  Dl-5. 

INBOUND  VERSUS  OUTBOUND . 

a.  Range  Errors.  Based  on  the  beacon-only  data,  flight  direction  does 
not  affect  the  variability  of  the  range  errors;  however,  the  "merged"  data 
indicate  that  the  "inbound"  error  variation  is  significantly  different  than 
the  "outbound"  error  variation.  Although  tests  of  significance  were  not  per- 
formed, due  to  the  high  variability  of  the  "outbound"  means  from  flight  to 
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TABLE  Dl-3.  AZIMUTH  ERROR  ESTIMATES  (DEGREES) 


A(x  - x)2' 


1 1 


TABLE  Dl-4.  COMBINED  AZIMUTH  ERROR  ESTIMATES1 


Flight 

Direction 

Radar 

Condition 

Pooled 

Standard 

Deviation 

sp 

(degrees) 

Grand 

Mean 

X 

(degrees) 

Tolerance  Limits 

X - 3Sp 
(degrees) 

X + 3Sp 
(degrees) 

X + 3S 
(6Sp)P 
(degrees) 

Inbound 

Beacon-Only 

0.1762 

N=lll 

l.ll3 

N=88 

0.58 

1.64 

1.06 

Merged 

.993 

N=97 

.54 

1.44 

.91 

Outbound 

Beacon-Only 

.1914 

N=89 

. 933 
N=98 

.36 

1.50 

1.15 

Merged 

. 1542 
N=135 

.853 

N=97 

.39 

1.31 

.92 

1 ARTS  III  target  reports  minus  EAIR  from  10  to  40  nmi  from  the  ASR-5. 

The  procedures  used  to  determine  homogeneity  of  standard  deviations  and  means 
are  given  in  appendixes  C3  and  C4,  respectively  (level  of  significance  - .01). 

1 All  4 days  were  combined. 

3 

November  18,  25,  and  December  4,  1974,  data  were  combined  (November  27  mean 
significantly  different). 

4 November  18,  27,  and  December  4,  1974,  data  were  combined  (November  25  standard 
deviation  significantly  different). 

Legend:  N = Combined  sample  size. 


Formulas 

r 2 2 2 

Sp  ^(Nl-DSl  + (N2-DS2  +....  + (Nk-l)Sk 
NX+  N2  +.T Nk-k 

X = NiX!  +N2X2  + +NkXk 

Nx  + N2+ '.Nk 
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TABLE  Dl-5.  RESULTS  OF  COMPARISON  TESTS 


1 


flight,  table  Dl-1  does  show  that  in  each  case,  the  "inbound"  means  were 
much  different  from  the  "outbound"  means  for  both  radar  conditions. 

b.  Azimuth  Errors.  For  both  radar  conditions,  error  variability  was 
independent  of  flight  direction.  However,  for  both  radar  conditions,  the 
"inbound"  and  "outbound"  means  differed  significantly. 

MERGED  VERSUS  BEACON-ONLY. 

a.  Range  Errors.  Based  on  the  "inbound"  data,  it  is  found  that  the 
radar  condition  does  not  affect  the  variability  of  the  range  errors,  however, 
the  "outbound"  data  indicates  that  the  variation  of  the  beacon-only  errors  was 
significantly  different  from  that  the  "merged"  errors.  For  the  mean  errors, 
the  "inbound"  data  did  show  significant  difference,  and  the  "outbound"  data 
from  table  Dl-1  show  show  that  for  each  flight,  the  "merged"  means  were  much 
different  than  the  "beacon-only"  means. 

b.  Azimuth  Errors.  For  both  flight  directions,  error  variability  is  not 
significantly  affected  by  radar  condition.  However,  for  both  flight  directions, 
the  "merged"  means  and  "beacon-only"  means  differed  significantly. 

SUMMARY  OF  RESULTS. 


Under  similar  operational  and  environmental  conditions  as  those  specified  in 
this  report,  the  estimated  ARTS  III  (RBTL)  target  report  accuracies  are  as 
follows : 

RANGE  ACCURACY.  Range  errors  should  not  vary  more  than  approximately  1,000 
feet  (+500  feet  of  the  mean  error) . The  inherent  variability  of  the  "merged" 
errors  appears  to  be  slightly  less  than  that  of  the  "beacon-only"  errors. 

Also,  error  variability  seems  to  be  generally  independent  of  flight  direction. 

The  range  of  mean  errors,  or  biases,  was  from  approximately  +0.2  nmi 
(+1,200  feet)  to  -0.1  nmi  (-600  feet).  On  the  average,  the  "merged"  errors 
were  always  less  (more  negative)  than  the  "beacon-only"  errors.  Comparing 
flight  directions,  the  "inbound"  errors  were  also  always  less  (more  negative 
on  the  average)  than  the  "outbound"  errors. 

AZIMUTH  ACCURACY.  Azimuth  errors  should  not  vary  more  than  approximately 
1°  (HK).5°  of  the  mean  error).  This  variability  does  not  seem  to  be  influenced 
significantly  by  either  radar  condition  or  flight  direction. 

The  range  of  mean  errors  or  biases  was  from  1.2°  to  0.8°.  On  the  average,  the 
"beacon-only"  errors  were  always  greater  (more  clockwise)  than  the  "merged" 
errors.  Comparing  flight  directions,  the  "inbound"  errors  were  also  always 
greater  (more  clockwise)  on  the  average  than  the  "outbound"  errors. 
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HISTOGSiMS  OF  FAHGE  «> 


azimuth 


ERRORS 


FIGURE  D1A-5 . HISTOGRAMS  OF  AZIMUTH  ERRORS  (MERCED') 


APPENDIX  DIB 


Z-TEST  FOR  THE  DIFFERENCE  BETWEEN  TWO  MEANS  (KNOWN  VARIANCES) 


PURPOSE. 

The  purpose  of  this  Z-test  is  to  determine  if  the  means  from  two  samples  are 
from  the  same  underlying  population,  given  that  the  population  variances  are 
known . 

PROCEDURE. 

The  two  populations  have  nj  and  n2  members,  sample  means  of  xq  and  x2,  true 
means  of  pq  and  y2»  and  known  variances  of  oq2  and  ^2^’  respectively. 

The  null  hypothesis  is  that  the  difference  between  the  true  means  is  equal  to 
zero;  i.e.  - H0;  p q - p2  = 0 

The  null  hypothesis  is  tested  by  means  of  the  Z statistic: 


The  Z value  obtained  from  the  above  is  entered  into  a standard  Z table,  or 
table  of  normal  deviates,  and  the  corresponding  value  (o/2)  obtained.  If 
this  is  lower  than  the  critical  o/2  value  agreed  upon,  then  the  null  hypothesis 
that  the  differences  of  the  sample  means  equal  the  differences  of  the  true 
means  is  not  contradicted  by  the  data.  The  o/2  significance  level  is  used, 
because  this  is  a two-sided  test. 

ASSUMPTIONS . 

This  analysis  assumes  that  the  two  samples  are  independent;  however,  they  need 
not  be  normally  distributed,  since  the  Central  Limit  Theorem  states  that  the 
means  of  samples  (N  ">10)  from  a population  are  normally  distributed,  and 
likewise  the  differences  between  them. 
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APPENDIX  D2 

STATISTICAL  ANALYSIS  OF  ARTS  111  (RBTL)  TARGET  REPORT  RANGE 
AND  AZIMTUH  ERRORS  (THEODOLITE  TRACKING) 

, by  ohn  J.  Wojciech,  ANA-140  (July,  1975) 

INTRODUCTION. 

Two  days  of  flights  were  conducted  at  NAFEC  to  determine  the  range  and 
azimuth  accuracy  of  the  ARTS  III  RBTL/ASR-5  system  using  the  Theodolites. 

Two  NAFEC  aircraft  were  used  to  fly  several  runs  of  the  flight  profile  shown 
in  figure  D2-1.  Table  D2-1  gives  a summary  of  the  general  flight  conditions 
including  Theodolite  accuracy  data  which  vary  depending  on  aircraft  position. 
Specific  design  and  test  condition  data  concerning  the  ARTS  III/ASR-5  system 
including  the  RBTL  computer  program  will  be  included  in  the  final  report  for 
activity  142-177-040. 

ERROR  SUMMARIES. 


ARTS  III  target  report  data  are  unpacked  using  a program  developed  by  the 
Data  Processing  Branch,  ANA-550.  A sample  copy  of  the  output  is  shown  in 
figure  B-2.  The  ARTS  III  tape  is  then  merged  with  the  Theodolite  data  tapes, 
one  for  each  Theodolite  pair,  to  provide  a summary  of  the  range  and  azimuth 
errors  as  shown  in  figure  D2-2.  Specifically,  Theodolite  range  and  azimuth 
values,  translated  and  rotated  to  those  of  the  ASR-5  (x=103,651.8  feet; 
y=107,858.8  feet;  Z=10,100  feet;  Rotation=307 . 7°) , were  subtracted  from 
"time-correlated"  ARTS  III  target  reports.  The  program,  also  developed 
by  ANA-550,  merges  only  good  quality  beacon  reports  (mode  3/A  and  mode  C 
validity  = 3;  strong  target  indicator  = 1).  This  was  done  for  several 
reasons,  the  primary  one  being  that  over  90  percent  of  the  beacon  data 
was  of  good  quality.  There  was  no  filtering  initially  applied  with  respect 
to  the  quality  of  the  radar  reports,  since  this  did  vary  significantly  during 
the  testing.  Finally,  multiple  targets  within  the  same  scan  caused  by  reflec- 
tions or  "ring-around"  were  also  not  processed. 


ERROR  DATA  GROUPING. 

In  an  effort  to  maintain  homogeneous  samples,  the  range  and  azimuth  errors 
were  grouped  by  azimuth  (300°-320°,  200°-220°,  20°-40°,  and  100°-120°),  test 
date  (November  6 and  8,  1974),  beacon  code  (1211  and  0722),  flight  direction 
(inbound  and  outbound)  and  radar  condition  (beacon-only  and  merged) . 

ERROR  DATA  SCREENING  AND  DISTRIBUTION  APPROXIMATION. 


The  errors  within  each  analysis  group  were  checked  for  reasonableness.  There 
were  a couple  of  obvious  cases  where  tracking  was  lost.  These  values  and  those 
few  that  failed  to  meet  the  "outlier  test"  ("Quality  Control  and  Industrial 
Statistics,"  by  A.  J.  Duncan,  Richard  D.  Irwin,  Inc.,  1965,  page  639)  were 
removed  from  the  analysis.  The  resultant  errors  for  each  data  group  were 
normally  distributed. 
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TABLE  D2-1 . FLIGHT  SUMMARY  (11/6/74  & 11/8/74) 


Beacon 

Altitude^ 

Theodolite 

Magnetic 

Maximum 

Aircraft 

Azimuth 

Range 

ID 

Code 

(Feet) 

Pair 

Sector 

(°) 

Error 
(Feet ) 2 

N]  03 

0722 

3,500 

P29/P13 

20°-40° 

8 to  300 

200°-220° 

60  to  450 

P8/P13 

300°-320° 

55  to  550 

100°-120° 

45  to  500 

N377 

1211 

3,000 

P36/P8 

20°-  40° 

13  to  200 

(11/6/74) 

200°-220° 

85  to  450 

N376 

P29/P36 

300°-320° 

4 to  400 

(11/8/74) 

100°-120° 

85  to  625 

^•Groundspeeds  generally  varied  between  175  and  200  knots. 

^Accuracy  data  were  calculated  using  the  method  described  on  page  11-2-7 
of  FAA  Publication,  "Technical  Facilities  at  NAFEC",  dated  July  1,  1969. 
Botl  minimum  and  maximum  distances  from  the  baselines  were  considered, 
and  an  "e"  value  of  43.49x10-5  corresponding  to  magnetic  tape  readings 
was  used. 
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curl:  :)2-2.  SAMPLE  LISTING  01  MERGED  ARTS  III  AND  THEODOLITE  DATA 


ERROR  PARAMETER  ESTIMATION. 

Based  on  the  information  contained  in  the  ANA-140  study  wherein  ARTS  III  and 
EAIR  were  compared  (appendix  Dl) , there  is  no  significant  linear  relationship 
between  the  range  and  azimuth  error  values  and  the  distance  of  the  aircraft 
from/to  the  ASR-5.  Therefore,  the  error  values  within  each  data  group  were 
lumped  together,  and  maximum-likelihood  estimates  were  computed. 

RANGE  ERROR  ESTIMATES.  Estimates  of  the  mean  and  standard  deviation  (vari- 
ability) of  the  range  errors  for  each  of  the  error  data  groups  are  shown  in 
table  D2-2.  Minus  values  of  mean  range  error  indicate  that  the  ARTS  III 
reported  data  were,  on  the  average,  less,  or  closer  to  the  ASR-5,  than  that 
recorded  by  the  Theodolites.  Conversely,  plus  mean  values  indicate  that  ARTS 
III  measurements  were,  on  the  average,  greater,  or  further  from  the  ASR-5,  than 
those  of  the  Theodolites.  To  determine  if  some  of  the  data  could  be  combined 
to  obtain  better  overall  estimates  of  range  accuracy,  tests  of  homogeneity 
were  performed,  and  where  allowable,  combined  estimates  were  computed. 

a.  Pooled  Standard  Deviations.  The  statistical  test  procedures  shown 
in  appendix  C3  were  applied  to  determine  if  the  standard  deviations  estimated 
for  each  azimuth  and  beacon  code  could  be  assumed  to  be  from  the  same  normal 
distribution.  Classification  was  by  flight  direction,  radar  condition,  and 
test  date.  In  each  case,  the  standard  deviations  can  be  assumed  co  be  from 
the  same  normal  distributions.  Therefore,  pooled  standard  deviations  were 
computed  as  shown  in  table  D2-3. 

In  addition,  the  range  of  errors  based  on  the  3-sigma  limits  is  given.  The 
range  of  +3-sigma,  or  6 sigma,  infers  that  at  least  99  percent  of  the  errors 
can  be  expected  to  be  within  the  range  derived. 

b.  Grand  Means.  The  statistical  test  procedures  shown  in  appendix  C4 
were  used  to  determine  if  the  means  estimated  for  azimuth  and  beacon  code 
could  be  assumed  to  be  from  the  same  normal  distribution.  For  both  azimuth 
and  beacon  code,  significant  differences  were  determined  at  the  .01  signifi- 
cance level.  Therefore,  these  data  could  not  be  combined. 

AZIMUTH  ERROR  ESTIMATES . Estimates  of  the  mean  and  standard  deviation 
(variability)  of  the  azimuth  errors  for  each  of  the  error  data  groups  are 
shown  in  table  D2-4.  Plus  mean  values  indicate  that  the  ARTS  III  reported 
data  were,  on  the  average,  greater,  or  more  clockwise,  than  that  reported  by  the 
Theodolites.  Again,  tests  of  homogeneity  were  performed  to  determine  if  some 
of  the  data  could  be  combined  to  provide  a more  precise  estimate  of  azimuth 
accuracy.  Combined  estimates  are  shown  in  table  D2-4. 

a.  Pooled  Standard  Deviations.  The  statistical  test  procedures  used 
previously  for  the  range  error  standard  deviations  were  applied  to  the  azimuth 
error  standard  deviations.  Pooled  standard  deviations  are  shown  in  table  D2-5. 
In  some  cases,  all  of  the  standard  deviations  were  pooled,  and  in  others, 
most  of  the  values  were  combined.  In  two  cases,  however,  only  the  1211  beacon 
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TABLE  D2-2.  RANCH  ERROR  ESTIMATES 


TABLE  D2-3.  COMBINED  RANGE  ERROR  ESTIMATES1 


Flight 

Direction 

Radar 

Condition 

Test 

Date 

Pooled 

Standard 

Deviation1 

SP 

(nmi) 

Range1 

X+3Sp=6Sp 

(nmi) 

Inbound 

Beacon-Only 

11/6/74 

0.018 

N=51 

0.108 

11/8/74 

.026 

N=120 

.156 

Merged 

11/6/74 

.026 

N=57 

.156 

11/8/74 

.033 

N=81 

.198 

Outbound 

Beacon-Only 

11/6/74 

.023 

N=d4 

.138 

11/8/74 

.027 

N=142 

.162 

Merged 

11/6/74 

.022 

N=47 

.132 

11/8/74 

.032 

N=53 

.192 

iStandard  deviations  tested  at  the  .01  level  of  significance. 


2A11  sample  estimates  were  pooled. 


^At  least  99  percent  of  the  range  errors  can  be  expected  to  fall  within 
this  range. 


Legend:  N - Sample  Size 


(Ni-l)Si2  +...+  (Nk-l)Sk 
N1  + n2  +. • •+Nk-k  * 
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TABLE  U2- 4.  AZIMUTH  ERROR  ESTIMATES 1 
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TABLE  D2-5.  COMBINED  AZIMUTH  ERROR  ESTIMATES 


Flight 

Direction 

1 

Radar 

Condition 

Test 

Date 

Pooled 

Standard 

Deviation 

SP 

(degrees) 

Range ^ 

X + 3S  = 6Sp 
(decrees) 

Inbound 

Beacon-Only 

11/6/74 

1.4Q 

11/8/74 

B 

1.61 

Merged 

11/6/74 

1.08 

11/8/74 

.1414 

N=79 

.85 

Outbound 

Beacon-Only 

11/6/74 

. 2165 

N =70  j 

1.30 

11/8/74 

fl 

87 

Merged 

11/6/74 

■BH 

1.14 

11/8/74 

— 

1.26 

^Standard  deviations  tested  at  the  .01  level  of  significance. 

^At  least  99  percent  of  the  azimuth  errors  can  be  expected  to  fall  within 
this  range. 

^Only  the  "1211"  standard  deviations  were  pooled. 

^All  sample  estimates  were  pooled. 

^Does  not  include  the  "1211/30°"  standard  deviation. 

^Does  not  include  the  "0722/110°"  standard  deviation. 

7 Only  the  "0722"  standard  deviations  were  pooled. 

Legend:  N - Sample  Size  Formula:  Sp=  j (N^-l)S-^^+.  . . + (Nj<-l)S|c^ 

^ " Ni  + N2  +. . .+Nk  -k 
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code  standard  deviations  could  be  pooled,  and  in  one  other  case,  only  the  0 l'i. 
beacon  code  standard  deviations  were  pooled.  Table  D2-5  also  contains  the 
ranges  within  which  at  least  99  percent  of  the  azimuth  errors  can  be  expo.  t< 
to  lie. 

b.  ' Grand  Means.  Using  the  statistical  procedures  referenced  above  t 
the  range  errors,  the  means  of  the  azimuth  errors  were  compared.  As  in  the 
case  of  the  range  errors,  significant  differences  were  determined  at  the  .01 
significance  level  for  both  azimuth  and  beacon  code.  Therefore,  grand  me. ms 
for  combined  means  were  not  computed. 

COMPARISON  TESTING . 

Based  on  the  statistical  test  procedures  described  in  appendix  DIF,  the  rant" 
and  azimuth  error  pooled  standard  deviations  given  in  tables  D2-3  and  D2-5, 
respectively,  were  compared  (using  Wang  700C/701  computer  programs)  to  deter- 
mine the  effects  of  flight  direction  and  radar  condition.  The  results  of 
these  analyses  are  shown  in  table  D2-6.  Due  to  the  absence  of  sufficient  mean 
error  data,  the  range  and  azimuth  mean  errors  were  not  statistically  compare* 
Visual  comparisons  of  the  means  from  tables  D2--2  and  D2-4  were  made,  however 

INBOUND  VERSUS  OUTBOUND . 

a.  Range  Error . Table  D2-6  indicates  that  there  is  no  significant 
difference  between  the  inbound  and  outbound  error  variability  for  both  the 
"beacon-only"  and  "merged"  conditions.  Table  D2-2  does  indicate  that  the 
"inbound"  means  were  always  less  or  more  negative  than  the  "outbound"  means, 
regardless  of  the  radar  condition. 

b.  Azimuth  Error.  The  one  case  tested  (November  8,  1974,  beacon-only; 
indicates  that  there  was  a significant  difference  in  the  "inbound"  and 
"outbound"  azimuth  error  variability.  Table  D2-4  indicates  that  most  of  the 
"inbound"  standard  deviations  are  greater  than  those  for  "outbound." 

Table  D2-4  also  indicates  that  the  "inbound"  means  were  generally  higher  or 
more  cLockwise  than  the  "outbound"  means,  regardless  of  the  radar  condition. 

BEACON-ONLY  VERSUS  MERGED. 


a.  Range  Error.  Table  D2-6  indicates  that  there  is  generally  no 
significant  difference  between  the  "beacon-only"  reports  and  "merged"  report 
variability.  Table  D2-2  does  indicate  that  the  "beacon-only"  means  were 
almost  always  greater  or  more  positive  than  the  "merged"  means,  regardless 
of  the  flight  direction. 


b.  Azimuth  Error.  The  one  case  tested  (November  8,  1974,  "inbound") 
indicates  that  there  is  a significant  difference  in  the  "beacon-only"  and 
"merged"  azimuth  error  variability.  Table  D2-4  indicates  that  most  of  the 
"beacon-only"  standard  deviations  are  greater  than  those  for  "merged". 

Table  D2-4  also  indicates  that  the  "beacon-only"  means  were  generally  higher 
or  more  clockwise  than  the  "merged"  means,  regardless  of  the  flight  direction. 
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TABLE  D2-6 . RESULTS  OF  STANDARD  DEVIATION  COMPARISON  TESTS 


S UMMARY  OF  RESULTS . 


Based  on  the  Theodolite  data  contained  in  this  report,  the  ANA-140  Technical 
Bulletin  dated  May  1975,  where  EAIR  data  was  compared,  and  the  Mitre  Report 
dated  May  28,  1975,  which  used  both  SAFI  and  EAIR  data,  the  following  AR'I  S 111 
(RBTL)  target  report  accuracies  can  be  expected  under  good  beacon  signal 
conditions : 

1.  RANGE  ACCURACY.  Table  D2-7  shows  the  range  error  mean  and  standard 
deviation  estimates  associated  with  the  three  independent  studies  mencioned 
above.  The  table  indicates  that  there  is  general  agreement  between  the  error 
estimates.  Generally,  the  mean  errors  were  within  +0.1  nmi,  with  standard 
deviations  between  0.02  and  0.03  nmi.  In  addition,  at  least  99  percent  of  the 
individual  range  errors  can  be  expected  to  vary  less  than  approximately 

0.16  nmi  (1,000  feet)  or  within  500  feet  of  the  mean  error. 

The  mean  errors  were  affected  by  both  radar  condition  and  flight 
direction.  "Beacon-only"  means  can  be  expected  to  be  less  (more  negative) 
than  the  "merged"  means.  "inbound"  means  can  be  expected  to  be  less  (more 
negative)  than  the  "outbound"  means. 

Range  error  variability,  based  primarily  on  the  ANA-140  study  where 
EAIR  was  used,  seems  to  be  independent  of  flight  direction,  but  may  be  slight 1\ 
less  under  "merged"  conditions  than  "beacon-only"  conditions.  The  EAIR 
analysis  is  considered  to  be  the  most  accurate  of  the  three  analyses  due  to  the 
comprehensiveness  of  the  data  collected  and  the  fact  that  the  EAIR  measure- 
ment variability  remains  constant  throughout  the  flights. 

2.  AZIMUTH  ACCURACY . Table  D2-8  shows  the  azimuth  error  mean  and 
standard  deviation  estimates  associated  with  the  three  studies.  Again,  there 
seems  to  be  general  agreements  between  the  results.  Generally,  the  mean  errots 
were  within  0.5°  and  I 3°,  with  standard  deviations  between  0.15°  and  0.25°. 

In  addition,  at  least  99  percent  of  the  individual  azimuth  errors  can  be 
expected  to  vary  less  than  approximately  1°  or  within  0.5°  of  the  mean  error. 

The  mean  errors  were  affected  by  both  radar  condition  and  flight 
direction.  "Beacon-only"  means  can  be  expected  to  be  higher  (more  clockwise) 
than  the  "merged"  means.  "Inbound"  means  can  be  expected  to  be  higher  (more 
clockwise)  than  the  "outbound"  means. 

Azimuth  error  variability  based  primarily  on  the  ANA-140  study  seems 
to  be  independent  of  flight  direction  and  radar  condition. 
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TABLE  D2-8.  SUMMARY  OF  AZIMUTH  ERROR  ESTIMATES  (Degrees) 


APPENDIX  D3 


ARTS  III  (RBTL)  REPORT/TRACK  ACCURACY 
by  John  J.  Wojciech,  ANA-140  (June,  1975) 


1.  On  December  4,  1974,  NAFEC  aircraft,  N377,  flew  several  radial  flights 
between  120°  and  140°  about  10  to  40  nmi  from  the  ASR-5.  Both  ARTS  III  RBTL 
target  report  and  tracking  message  data  were  extracted  as  shown  in  figure  B-2. 
These  data  were  then  compared  to  corresponding  EAIR  data  as  shown  in  figure  9. 
It  should  be  noted  that  each  of  the  target  reports  was  based  on  good  quality 
beacon  (mode  3/A  and  C validities=3)  and  that  each  of  the  tracking  messages 
was  also  of  high  quality  (f irmness=39; . 

2.  The  target  report  errors  were  broken  down  by  flight  direction  (inbound/ 
outbound)  and  radar  condition  (beacon-only /merged) • The  tracking  message 
values  were  subdivided  by  flight  direction.  Each  group  of  errors  was  checked 
for  "outliers"  and  the  distribution  of  the  errors  was  found  to  be  generally 
normal.  The  mean  and  standard  deviations  associated  with  each  error  group  are 
shown  in  table  D3-1.  A minus  mean  error  in  range  indicates  that  ARTS 
measured,  on  the  average,  less  than  EAIR.  A plus  mean  error  in  azimuth 
indicates  that  ARTS  read,  on  the  average,  more  clockwise. 

3.  Target  report  and  tracking  message  accuracy  were  compared  statistically 
using  the  parameter  estimates  given  in  table  D3-1.  The  results  of  these 
comparison  tests,  shown  in  table  D3-2,  indicate  that  range  error  variability 
is  less  using  the  target  report  data;  however,  there  do  not  seem  to  be  any 
obvious  differences  in  the  means  of  the  range  errors.  In  the  case  of  the 
azimuth  data,  there  do  not  seem  to  be  any  real  differences  in  either  the 
error  means  or  the  standard  deviations  of  the  errors. 
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Table  D3-1  Target  Report  and  Tracking  Message  Error  Es-ima* 

Target  Report 


Error 


Flight 

Direction 


Merged 


T- 

N ! 


— -4 


Beacon-Only 

~T“ — 1 


Track l 
Messar 


N 


! 


Range 
( nmi ) 


Inbound  30  -.099  .019132  !-.061  .028  32  -.11 


Outbound  :37  ! —.  0671  . 01 2 i 3 1 ; —.  038  .030  3 0 -.  0 c 


1-~ 


.1-  — 


— 

I 


+ 


Azimuth 
(Deg . ) 


Inbound  ; 30  ,93  ,13932  1.05 


.139  32 


. 9 ( 


-f 


Outbound  37 


,88 


I 


.120' 31 


.97 


.144  30 


o ; 


ARTS  I I i targe*  reports/*racking  messages  minus  EATR  rror 
40  nmi  from  the  ASR-5. 


Legend: 


M 

7 

n 

O 


Sample  size 
Mean 

Standard  Deviation 


Formulas : 


S 
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Table  D3-2.  Results  of  Comparison  Tes^s1 

Targe*  Repor*  vs.  Tracking  Message 


; Parameter 


Inbound 
1 


Outbound 

■ r 


Merged 

Beacon-Only 

Merged 

JSeacon-r'n]  y 

Range 

Standard 

Deviation 

Signif ican* 
( .01 ) 

i' 

Signif ican* 

( .01) 

significan* 
( .01 ) 

No* 

Si  gni  f i car. * 
( .05) 

Mean 

| 

Mo* 

Significan* 

(.05) 

Signif ican* 
( .01) 

No* 

Signif ican* 
( . 05) 

No* 

Signif  ica  r.* 
( .01  ) 

Azimuth 

Standard 

Deviation 

■ 

Not 

Significan* 



Not 

Significan* 
X.JJ5)  .... 

No* 

Signif ican* 

. ( .05)  .... 

No* 

Signi^  icar.* 

______ 

Mean 

No* 

Signif ican* 1 
( .05) 

1 

l 

1 

Significan*, 
( .01) 

Mo* 

Significan* 
( .05) 

S i r n i f i c a : . * 
( .01  ) 

Standard  deviation  comparison  test  procedures  can  be  found  or. 
Page  511  of  "Ouali*y  Control  and  Industrial  S*a*is*ics"  by 
Acheson  J.  Duncan,  Richard  D.  Irwin,  Inc.,  1065.  Mean  cornarisor. 
procedures  can  be  found  on  Page  505  of  *he  sane  *ex*. 


Legend:  ( ) - Level  of  Significance. 
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APPENDIX  DA 

ARTS  III  PADAR  DISPLAY  ERROR  ANALYSIS 
by  John  J.  Wojciech,  ANA-140  (January  1975) 


SUMMARY. 


A test  was  conducted  at  the  National  Aviation  Facilities  Experimental  Center 
(NAFEC)  in  Atlantic  City,  New  Jersey,  to  determine  the  ARTS  III  digital  symbol 
position  error.  Data  were  collected  in  all  four  quadrants  and  over  most  of 
the  11-inch  display  radius.  The  NAFEC  Telereadex  Film  Reader  was  used  to 
measure  the  position  of  the  digital  symbol  from  scope  photographs.  These 
target  positions  were  compared  to  those  generated  by  the  ARTS  III  to  determine 
the  display  errors  both  in  range  and  azimuth. 

The  range  and  azimuth  errors  were  showed  to  be  definitely  correlated  with  the 
distance  of  the  symbol  from  the  display  center.  Based  on  a simple  linear 
regression  analysis,  the  mean  range  and  azimuth  errors  can  be  expected  to  fall 
within  +0*15  nmi.  Individual  range  and  azimuth  errors  can  be  expected  to  fall 
within  +0.25  nmi,  95  percent  of  the  time. 

The  report  shows  that  these  display  errors  become  insignificant  compared  to 
the  reading  errors  that  are  possible  using  the  human  eye  as  the  display 
range  setting  is  increased  beyond  30  nmi. 


INTRODUCTION 
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PURPOSE. 

This  technical  bulletin  describes  the  analysis  performed  at  the  National 
Aviation  Facilities  Experimental  Center  (NAFEC) , Atlantic  City,  New  Jersey, 
to  determine  the  range  and  azimuth  errors  of  an  ARTS  III  radar  display. 

BACKGROUND. 

The  Systems  and  Equipment  Engineering  Branch,  ANA-140,  has  been  tasked  to 
investigate  the  errors  associated  with  an  ARTS  III  Terminal'  Radar  System, 
Subprogram  No.  142-177.  This  system  consists  of  three  major  parts;  the 
airport  surveillance  radar  (ASR)  with  associated  beacon,  the; ARTS  III,  and  tin 
radar  display.  The  analysis  contained  herein  deals  with  the  RBTL  configurati 
radar  display  error  component  of  the  system  error  and  is  defined  as  the 
difference  between  the  position  of  a target  as  determined  by  ARTS  III  and  th. 
position  of  the  digital  symbol  representing  that  target  on  the  display. 

The  ARTS  III  determines  the  range  and  azimuth  of  a target  and  records  this 
information  on  magnetic  tape  (UNISERVO  Vic  system).  Simultaneously,  the 
x and  y coordinates  of  the  target  are  sent  to  the  radar  display.  This  digital 
data  are  then  converted  to  analog  signals  and  are  displayed  on  the  cathode  ray 
tube  (CRT).  There  is  approximately  a 1-second  delay  between  the  time  the 
target  is  reported  and  the  time  the  digital  symbol  is  positioned.  This  time 
is  required  in  the  event  a radar  target  is  not  received  and  a predicted  target 
generated  by  the  software  has  to  be  displayed. 


DISCUSSION 


TEST  CONDITIONS. 

DISPLAY . The  Type  1 Vertical  Display  used  in  the  analysis  is  similar  to  the 
one  shown  in  figure  D4-1.  The  CRT  is  22  inches  in  diameter  and  can  display 
both  analog  beacon  and  broadband  radar  from  the  ASR,  and  alphanumeric  or  digi  . > 
data  generated  by  the  ARTS  III.  Display  ranges  can  be  selected  in  1-  and  2-nmi 
increments  from  6 nmi  to  60  nmi.  The  specific  display  and  ARTS  III  system 
tested  are  part  of  the  Terminal  Automation  Test  Facility  (TATF)  at  NAFEC. 

DISPLAY  CALIBRATION.  Prior  to  the  test,  the  radar  coordinate  positioning 
accuracy  of  the  display  was  checked  by  the  UNIVAC  maintenance  technician  in 
accordance  with  the  procedures  recommended  for  field  use.  These  procedures 
(ARTS  III  DAS/DEDS  PDFA,  Operating  Procedures,  Section  4.4,  01-2749-A, 

April  15,  1971)  indicate  that  adjustments  are  only  required  if  the  difference 
between  the  actual  position  of  the  center  of  the  displayed  digital  symbol  and 
the  computer-generated  position  exceeds  0.125  nmi.  This  check  is  done  visually 
at  the  12-nmi  range  setting  using  range  marks  of  2 nmi  and  a symbol  size  of 
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"2".  This  smaller  range  is  selected  because  it  is  extremely  difficult  to 
visually  distinguish  0.125  nmi  at  the  larger  ranges.  Figure  04-2  shows  the 
exact  relationship  between  coordinate  p>  error  or  'display  error" 

display  distance  for  various  range  se I inch  CRT  display.  At 

the  12-pmi  range  setting,  an  error  t ■ equal  to  0.114  inches,  whil 

at  the  60-nmi  range,  this  error  r.  >rest- • 0.  23  inches. 


TEST  PROCEDURES.  On  March  8,  197-, 
aircraft  using  the  flight  profile- 
display  range  setting  was  et  to  the  . 
viewing  of  the  flights  and  yet  p.  > - : :•  \.i> 


i'  ! were  made  by  a NAFEC 
tures  D4-3  and  D4-4 . The 
/ a 1 u e that  would  provide  complete 
t .rget  reading  accuracy. 


The  14-nmi  range  setting  was  selected  al  -nr.t  range  marks  and  a s 

size  of  "2."  It  should  be  noted  that  nT  one  range  setting  was  investigated, 
since  the  inherent  display  error  is  a function  of  the  sweep  center  and  linear  it- 
adjustments.  A change  in  range  setting  does  not  affect  these  adjustments.  Kan., 
setting  only  affects  the  reading  accuracy  as  discussed  previously. 

The  ARTS  III  aircraft  position  data  or  the  display  input  data  were  recorded 
on  magnetic  tape  each  scan  of  the  radar  or  every  four  seconds  using  the 
UNISERVO  Vic  system  at  the  TATF.  The  digital  symbol  positions  indicated 
on  the  display  were  recorded  on  photographic  film  at  the  rate  of  one  frame 
per  second.  A typical  scope  camera  installation  is  shown  in  figure  D4-5. 

DATA  REDUCTION. 


DATA  SAMPLE.  Data  points  were  selected  from  each  quadrant  and  from  various 
distances  or  ranges  from  the  ASR  from  about  3 nmi  out  to  about  11  nmi.  At 
the  14-nmi  range,  this  distance  corresponds  to  about  2.5  inches  out  to  about 
8.5  inches  from  the  center  of  the  CRT.  The  number  of  data  points  selected 
was  somewhat  restricted  by  the  use  of  the  scope  photos.  It  took  approximate  1 
10  minutes  to  accurately  read  each  aircraft  position,  including  locating 
the  frame  of  interest  and  doing  a calibration  of  the  film  reader. 

ARTS  III  DATA.  The  ARTS  III  magnetic  tape  was  reduced  on  the  7090  computer 
using  a program  developed  by  the  Data  Processing  Branch,  ANA-550.  A sample 
copy  of  a target  report  listing  showing  target  range  and  azimuth  is  shown 
in  figure  D4-6.  Multiple  target  reports  within  the  same  scan  were  not  used 
in  the  analysis  to  preclude  the  possibility  of  an  error  in  correlating  the 
magnetic  tape  and  the  scope  photo  targets. 

SCOPE  PHOTOS . The  NAFEC  Telereadex  Film  Reader  shown  in  figure  D4-7  was  u ed 
to  determine  the  x-y  coordinates  of  the  center  of  the  aircraft  targets  select 
for  the  analysis.  These  coordinates,  considered  accurate  to  +0.2  nmi,  were 
converted  to  range  and  azimuth  values  using  the  Wang  700C  computer.  The 
detailed  procedures  used  to  measure  the  target  position  from  the  scope  photos 
are  given  in  appendix  D4A. 
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FIGURE  D4-3 
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FIGURE  D4-7.  NAFEC  TELEREADEX  FILM  READER  EQUIPMENT 


ERROR  SUMMARIES. 


Tables  D4-1  and  D4-2  show  both  the  aircraft  range  and  azimuth  positions  as 
reported  and  displayed  by  the  ARTS  III  for  the  NE-SW  and  NW-SE  flight  profiles, 
respectively.  Also  shown  are  the  display  errors  or  the  differences  resulting 
in  subtracting  the  reported  positions  from  the  displayed  positions.  The 
linear  offsets  associated  with  the  azimuth  degree  errors  are  also  listed 
for  more  direct  comparison  to  the  range  errors.  These  error  summaries  were 
generated  using  a Wang  700C/701  computer  program  developed  by  ANA-140. 

ERROR  ANALYSIS. 


Since  the  inherent  display  errors  are  independent  of  the  range  selected,  dis- 
play error  was  analyzed  as  a function  of  the  physical  distance  of  the  target 
from  the  display  center  rather  than  a particular  radar  range.  Therefore, 
the  "displayed"  range  values  given  in  tables  D4-1  and  D4-2  were  multiplied 
by  the  associated  CRT  scale  factor  of  11  inches/14-nmi  range  (full  deflection). 
The  range  and  azimuth  errors  were  grouped  into  20°-50°,  115°-130°,  190°-215°, 
and  285°-315°  azimuth  sectors  with  respect  to  the  ASR-4  site.  These 
errors  were  plotted  individually  against  the  distance  from  the  display  center, 
and  it  was  concluded  that  both  range  and  azimuth  error  vary  linearly  with 
the  distance  from  the  display  center.  Simple  linear  regression  and  correlation 
analyses  based  on  the  method  of  least  squares  were  performed  to  quantify  this 
relationship.  A brief  description  of  these  analyses,  including  the  associated 
assumptions,  is  given  in  appendix  Cl.  The  analysis  results,  obtained  using  a 
Wang  700C/701  computer  program  developed  by  ANA-140,  are  given  as  follows: 

Range  Errors.  The  Y=a+bX  line  shown  in  figure  D4-8  for  each  azimuth 
sector  represents  the  best  estimate  of  the  universe  regression  line  or  the 
linear  relationship  between  range  error  (Y)  and  the  distance  from  the  display 
center  (X)  for  distances  between  2 inches  and  9 inches.  It  can  be  noted  that 
the  closer  the  slope  of  the  line  (b)  is  to  zero,  the  less  dependent  the  range 
error  is  on  the  distance  from  the  display  center.  A negative  slope  (b^.0) 
indicates  that  as  the  distance  from  the  display  center  increases,  the  range 
error  increases  in  the  negative  direction.  A positive  slope  (b  *?0)  indicates 
that  as  the  distance  from  the  display  center  increases,  the  range  error 
increases  in  the  positive  direction.  The  95-percent  prediction  limits,  also 
shown  in  figure  D4-8,  indicate  for  a given  distance  from  the  display  center, 
at  least  95  percent  of  the  individual  range  errors  can  be  expected  to  lie 
within  these  limits. 

Table  D4-3  summarizes  the  results  of  the  regression  and  correlation 
analyses  for  each  azimuth  sector.  The  coefficient  of  correlation  estimates, 
r,  associated  with  each  regression  line  estimate,  represents  a measure  of  the 
strength  of  the  linear  relationship  between  range  error  and  the  distance  of 
the  target  from  the  display  center,  or  how  well  the  associated  regression 
line  fits  the  data.  Plus  or  minus  values  of  r between  0.4  and  0.6  are  con- 
sidered indicative  of  fair  to  good  correlation,  while  coefficients  between 
0 and  0.4  indicate  little  or  no  correlation  and  coefficients  between  0.6  and 
1.0  indicate  a high  degree  of  correlation.  Table  D4-3  shows  that  for  the 
four  azimuth  sectors  combined,  the  best  estimate  of  range  error  between 
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TABLE  D4-2.  ARTS  III  REPORTED  VERSUS  ARTS  III  DISPLAYED  POSITION  (NW-SE) 


TABLE  D4-3.  RANGE  ERROR  VERSUS  AZIMUTH  SECTOR 


Azimuth 

Sector 

(°) 

Estimated 

Regression 

Line 

Y=a+bX 

(nmi) 

Estimated 
Regression 
Line  Range* 
(2-9  inches) 
(nmi) 

Estimated 
95%  Prediction 
Limit  Range** 
(2-9  inches) 
(nmi) 

BS9I 

■SO 

-0.069  - 0.0063X 
(r=-. 44) 

-0.08/-0.12 

-0.01/-0.20 

115-130 

(n=6) 

-.051  - . 0121X 
(r=  -.91) 

-.07/-. 16 

-.03/-. 20 

190-215 

(n=8) 

-.065  - . 0080X 
(r=  -.49) 

-.08/-. 14 

.01/-. 23 

285-315 

(n=15) 

-.093  - . 0087X 
(r=  -.45) 

-.11/-. 17 

-.04/-. 25 

Legend:  X - Distance  from  center  of  display 

Y - Error  that  can  be.  expected  at  X on  average 
a - Y-Intercept  (X=0)  of  the  regression  line 
b - Slope  of  the  regression  line 
n - Sample  size 

r - Coefficient  of  correlation  estimate 
* - Y at  X=2  inches  to  Y at  X=9  inches 
**  - A minimum  of  95  percent  of  the  errors  can  be  expected 

to  lie  within  these  values  over  the  2-  to  9-inch  distaii-  e 
from  the  display  center. 
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figure  DA-8.  range  error  versus  distance  from  the  display  center 
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2 inches  and  9 inches  from  the  display  center  is  from  -0.07  nmi  to  -0.17  nmi. 
Based  on  the  prediction  limit  data,  at  least  95  percent  of  the  individual 
range  errors  can  be  expected  to  lie  within  0.01  nmi  and  -0.25  nmi  over  the 
2-  to  9-inch  distance. 

[ I 

' Azimuth  Errors.  Figure  D4-9  shows  the  relationship  of  the  azimuth  error 

(y)  and  the  distance  from  the  display  center  (x)  for  each  azimuth  sector  and 
distances  between  2 and  9 inches  from  the  display  center.  Table  D4-4  sum- 
marizes the  results  of  the  regression  and  correlation  analyses  for  each 
azimuth  sector.  Combining  the  azimuth  sector  results,  the  best  estimate  of 
azimuth  error  between  2 inches  and  9 inches  from  the  display  center  is  from 
-0.07  to  0.09  nmi.  Based  on  the  prediction  limit  data,  at  least  95  percent 
of  the  individual  azimuth  errors  can  be  expected  to  lie  within  -0.16  nmi  and 
0.15  nmi  over  the  2-  to  9-inch  distance. 


AZIMUTH  ERRORS  (NMI) 


— . 1 


Y=a+t>x 


GO 

1| 


■£IiV «'-»“lcI,ON  UM‘T 


J I L 

2 4 6 


X 

8 


10 


.2h 


J I L_ 

6 8 10 


-J I I l 1 

2468  10 

DISTANCE  FROM  THE  DISPLAY  CENTER  (INCHES) 

76-6-D4-9 


FIGURE  D4-9. 


AZIMUTH  ERROR  VERSUS  DISTANCE  FROM  THE  DISPLAY  CENTER 
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TABLE  D4-4 . AZIMUTH  ERi.OR  VERSUS  AZIMUTH  SECTOR 


Azimuth 

Sector 

n 

Estimated 

Regression 

Line 

Y=a+bX 

(nmi) 

Estimated 
Regression 
Line  Range* 
(2-0  inches) 
(nmi) 

Estimated 
95 7.  Prediction 
Limit  Range** 
(2-9  inches) 
(nmi) 

20-50 

(n=ll) 

0.043  + 0.0052X 
(r=  .42) 

0.05/0.00 

-0.00/0.15 

115-130 

(n=6) 

.022  - . 0104X 
(r=  -.68 

.00/-. 07 

.10/-. 16 

190-215 

(n=8) 

-.019  - . 0007X 
(r=  -.06) 

-.02/-. 03 

.06/. 10 

285-315 

(n=15) 

-.054  + . 0128X 
(r=  .60) 

-.03/. 06 



-. 10/. 14 

Legend:  X 

Y 
a 
b 
n 
r 
* 

** 


Distance  from  center  of  display 
Error  that  can  be  expected  at  X on  average 
Y-Intercept  (X=0)  of  the  regression  line 
Slope  of  the  regression  line 
Sample  size 

Coefficient  of  correlation  estimate 
Y at  X=  2inches  to  Y at  X=9inches 

A minimum  of  95  percent  of  the  errors  can  be  expected 
to  lie  within  these  values  over  the  2 to  9 inch  distance 
from  the  display  center. 
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1.  Mean  range  and  azimuth  errors  associated  with  the  digital  symbol  of  an 
ARTS  III  display,  maintained  in  accordance  with  current  field  procedures,  can 
be  expected  to  fall  within  +0.15  nmi  from  2 inches  to  9 inches  from  the  display 
center.  Individual  range  and  azimuth  errors  over  this  same  distance  can  be 
expected  to  fall  within  +0.25  nmi,  95  percent  of  the  time. 

2.  Depending  on  the  range,  range  mark,  and  character  size  settings  and  the 
air  traffic  control  specialist's  (ATCS's)  visual  acuity,  the  errors  given 

in  paragraph  1 above  may  be  significant.  The  overriding  influence  is  considered 
to  be  the  range  setting.  Going  back  to  the  nomograph  given  in  figure  D4- 

2,  it  can  be  seen  that  an  error  as  high  as  0.25  nmi  represents  less  than 

0.1  inches  for  range  settings  greater  than  30  nmi.  An  error  of  0.15  nmi  would 
probably  be  undiscernable  at  the  30  nmi  or  greater  range  setting,  since  the 
associated  distance  would  be  0.06  inches  or  less.  It  is  interesting  to  note, 
that  if  the  ATCS  is  off  just  0.05  inches  in  estimating  the  digital  symbol 
position,  an  error  of  0.27  nmi  would  be  made  if  the  range  setting  was  60  nmi. 

3.  The  results  given  in  paragraph  1 above  are  considered  satisfactory  for 
use  in  estimating  the  overall  ARTS  III/ASR  digital  system  error  (project 
142-177).  Additional  scope  photo  data  reduction  and  analysis  is  not 
considered  necessary. 
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APPENDIX  D4A 


SCOPE  PHOTOGRAPH  POSITION  MEASUREMENT 


The  following  procedures  were  followed  to  determine  the  range  and  azimuth  of 
the  digital  symbol  from  the  scope  photographs: 

1.  An  overlay  was  used  to  find  the  center  of  the  display  range  marks  which 
were  photographed  in  their  entirety  during  the  test.  Range  marks  of  2 nmi  were 
selected  for  a display  range  of  14  nmi. 

2.  The  Telereadex  x and  y wires  were  positioned  to  intersect  at  the  center 
of  the  range  marks.  The  x and  y electronic  counters  were  then  set  to  zero. 

3.  The  film,  which  was  taken  at  a rate  of  one  frame  per  second,  was  advanced 
until  a frame  of  interest  appeared.  This  frame  was  at  least  2 seconds  or 
frames  following  the  time  that  the  ARTS  III  reported  the  target  position 

to  the  display.  This  time  was  allowed  for  updating  of  the  digital  symbol 
position . 

4.  To  obtain  a calibration  or  the  number  of  counts  per  mile,  the  x and  y 
wires  were  moved,  first  one,  then  the  other,  to  each  of  the  range  marks 
immediately  adjacent  to  the  digital  symbol.  Several  readings  of  the  x and  y 
counts  were  obtained  in  order  to  minimize  reading  error. 

5.  The  x and  y wires  were  then  positioned  to  intersect  the  center  of  the 
digital  symbol  and  the  x and  y counts  were  recorded.  Again  several  readings 
were  obtained  to  minimize  reading  error. 

6.  The  target  x and  y counts  were  converted  to  x and  y nautical-mile 
values  using  linear  interpolations  of  the  counts  per  nautical  mile  data  and 
target  count  data  derived  in  paragraphs  4 and  5 above,  respectively. 


